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EXECUTIVE  SUMMARY 


The  goal  of  the  research  was  the  development  of  a  mathematical  theory  of 
task-oriented  organizations,  where  the  tasks  are  primarily  information 
processing  and  decision  making. 

Information  theoretic  models  were  used  to  analyze  the  effect  of  different 
perceptions  of  uncertainty  and  relative  value  of  tasks  by  individual  decision 
makers  on  organizational  performance .  The  definition  and  evaluation  of 
timeliness  as  a  measure  of  effectiveness  of  command  and  control  systems  was 
also  addressed. 

A  measure  of  organizational  effectiveness  was  developed  for  the  case  in 
which  systems  designer  and  decision  maker  differ  in  their  perception  of  the 
uncertainty  associated  with  inputs  or  tasks .  This  measure  was  defined  as  the 
ratio  of  strategies  that  yield  satisficing  performance  to  the  total  admissible 
strategies.  A  hierarchical  and  a  parallel  three-person  organization  were  used 
to  illustrate  the  results.  The  complexities  of  assessing  the  timeliness  of  C3 
systems  were  addressed  through  the  extension  of  the  system  effectiveness 
methodology  to  consider  not  trnly  the  structure  and  protocol  of  a  C3  system, 
but  also  the  effect  of  doctrine  on  measures  of  effectiveness.  Again,  the 
theoretical  development  were  illustrated  through  application  to  a  generic  C* 
system. 

The  overall  research  effort  will  contribute  to  methodologies  for  the 
analysis,  design,  and  evaluation  of  command  and  control  systems  that  support 
tactical  Army  organizations. 


1 .  INTRODUCTION 


When  information  processing  and  decision  making  tasks,  which  have  to  be 
performed,  exceed  the  capabilities  of  a  single  decision  maker,  organizations 
are  formed.  The  functions  that  each  organization  member  is  allocated,  the 
interactions  among  the  members,  and  the  protocols  that  govern  these 
interactions  constitute  the  organizational  form  or  structure.  This  structure, 
in  turn,  affects  the  individual  member's  workload,  as  well  as  the  performance 
of  the  organization  as  a  whole. 

The  point  of  view  taken  in  this  report  is  that  of  the  organization 
designer.  Given  a  set  of  tasks,  the  designer  develops  an  organizational 
structure  and  then  evaluates  it  to  determine  if  it  meets  a  set  of  performance 
requirements.  If  it  does,  then  the  next  step  is  to  assess  the  sensitivity  of 
the  design  to  the  assumptions.  To  carry  out  the  assessments,  it  is  necessary 
that  a  methodology  be  developed  and  that  measures  of  effectiveness, 
appropriate  for  the  problem  at  hand,  be  developed. 

Both  of  these  steps  are  addressed  in  this  report.  They  are  imbedded  in  a 
broader  framework,  that  of  the  development  of  an  analytical  methodology  for 
the  design  and  evaluation  of  command  and  control  organizations.  Consequently, 
for  completeness,  elements  of  that  methodology,  which  form  the  basis  for  the 
specific  results  obtained  in  this  investigation,  will  be  presented.  First,  in 
Chapter  2.  the  basic  model  of  the  interacting  decisionmaker,  developed  by 
Boettcher  and  Levis  [1],  [2],  [3]  will  be  introduced  and  its  properties 
described . 

In  the  following  chapter,  the  model  of  organizations  performing  demanding 
tasks  under  time  contraints  is  presented  and  the  methodology  for  evaluating 
them  is  given  in  some  detail,  since  it  is  the  focus  of  the  research  carried 
out  under  this  contract.  In  developing  a  design,  the  designer  assumes  that  he 
knows  the  uncertainty  that  characterizes  the  tasks  to  be  performed  by  the 
organization  and  that  he  also  knows  the  value  of  each  task.  He  also  assumes 
that  the  organization  members,  in  processing  the  information  and  making 
decisions,  have  identical  knowledge  of  task  uncertainty  and  perception  of  task 
value.  In  general,  it  is  very  difficult  to  assess  the  probability 
distribution  of  the  tasks;  it  is  also  quite  unlikely  that  the  designer  and  the 
actual  members  of  the  organization  have  the  same  perception  of  the  tasks' 
probability  distribution.  In  this  chapter,  this  assumption  is  relaxed:  it  is 
assumed  that  the  designer  knows  the  tasks'  real  probability  distribution, 
while  the  organization  members  have  a  different  perception  of  this 
distribution.  The  second  assumptions  about  common  perception  of  task  value  is 
also  improbable:  the  decisionmakers  in  the  organization  may  assign  different 
value  to  the  various  tasks  from  the  ones  the  designer  assumed.  The  relaxation 
of  this  assumption  is  also  presented  in  Chapter  3 . 

In  order  to  pose  both  problems  properly,  two  additional  assumptions  must 
be  made.  The  first  is  trivially  true  in  practice,  but  needs  to  be  made 
explicit  in  the  mathematical  analysis:  there  is  no  communication  between  the 
designer  and  the  organization  members .  The  second  assuption  is  that  the 
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designer  knows  both  the  true  task  uncertainty  and  the  uncertainty  as  perceived 
by  each  of  the  organization  members,  as  well  as  both  the  true  value  of  the 
tasks  and  the  value  assigned  to  the  tasks  by  each  decisionmaker.  This 
information  is  needed  in  order  to  carry  out  sensitivity  analyses.  The 
theoretical  development  is  then  applied  to  two  three-person  organizations,  a 
hierarchical  and  a  parallel  one,  in  order  to  illustrate  the  sensitivity 
analysis  and  its  use  in  selecting  organizational  designs. 

The  same  effectiveness  analysis  methodology,  but  with  different  measures 
of  performance,  can  be  applied  to  the  evaluation  of  the  architecture  of 
command  and  control  systems .  Such  an  application  to  a  model  abstracted  from 
an  actual  US  Army  system  is  presented  in  Chapter  4. 

Finally,  in  the  concluding  chapter,  the  results  are  summarized  and 
suggestions  for  further  research  are  made . 


I*  lip ipuppp-pprvi'Ji '  IP 


2.  THE  INTERACTING  DECISIONMAKER  MODEL 

The  first  step  in  modeling  an  organizational  structure  is  the  modeling  of 
the  tasks  to  be  performed  by  the  organization.  The  second  step  is  to  develop 
an  appropriate  mathematical  model  of  the  organization  member.  Specifically, 
this  model  must  incorporate  provisions  for  the  variety  of  interactions  that 
can  exist  among  decisionmakers  in  an  organization.  These  two  steps  are 
discussed  in  this  chapter.  In  addition,  the  necessary  analytical  tools  are 
introduced,  namely,  Petri  Nets  and  N-dimensional  information  theory.  The 
former  is  used  to  describe,  rather  precisely,  the  architecture  of  the 
decisionmaking  model,  and  of  the  organizations,  while  the  latter  is  used  to 
model  the  cognitive  workload  of  the  individual  decisionmakers. 

2.1  PETRI  NETS 

In  this  work,  only  the  basic  properties  of  Petri  Nets  are  needed  to 
describe  the  models.  In  related  work  for  the  Office  of  Naval  Research  and  for 

the  Technical  Panel  on  C1  of  the  Joint  Directors  of  Laboratories,  several 

measures  of  performance  (MOPs)  of  organizations  have  been  obtained  using  some 
more  advanced  concepts  from  Petri  Net  theory  [4,5].  For  an  introductory 
treatment  of  Petri  Nets  as  modeling  tools,  the  text  by  Peterson  [6]  is 
recommended . 

Petri  Nets  are  bipartite  directed  graphs  reprsented  by  a  quadruple 

(P,T,I,0) .  By  convetion,  P  is  the  set  of  one  type  of  nodes,  called  places  or 
circle  nodes,  and  T  is  the  set  of  the  second  type  of  nodes,  called  transitions 
or  bar  nodes.  Places  can  depict  the  presence  of  signals  or  represent 

conditions;  transitions  can  depict  processes  or  events.  Consequently,  the 
arcs  that  connect  the  nodes  that  form  the  graph  can  only  go  from  one  type  of 
node  to  another  -  either  from  a  place  to  transitions,  or  from  a  transition  to 
places.  The  mapping  I  corresponds  to  the  set  of  directed  arcs  from  places  to 
transitions,  i.e.,  it  defines  the  input  places  of  the  transitions,  while  the 
mapping  0  corresponds  to  the  set  of  directed  arcs  from  transitions  to  places; 
i.e.,  it  defines  the  output  places  of  each  transition.  For  ordinary  Petri 
Nets  -  the  only  type  considered  here  -  the  mappings  I  and  0  take  values  from 

the  closed  set  {0,1};  1  denotes  the  presence  of  a  link  between  two  nodes, 

while  0  denotes  the  absence . 

A  Petri  Net  consisting  of  four  transitions  and  five  places  is  shown  in 

Figure  2.1.  Tokens,  denoted  by  dots  in  places  or  circle  nodes,  control  the 

execution  of  a  Petri  Net.  A  marking  of  a  Petri  Net  is  a  mapping  which  assigns 
a  non-negative  integer  number  of  tokens  to  each  place  of  the  net.  Since  the 
number  of  tokens  in  a  place,  in  general,  is  not  bounded,  there  can  be  an 
infinite  number  of  markings  associated  with  each  net.  A  Petri  Net  is  said  to 
execute  when  a  transition  fires.  A  transition  can  fire,  only  if  it  is 
enabled .  For  a  transition  to  be  enabled,  all  its  input  places  must  contain  at 
.  least  one  token  each.  When  a  transition  fires,  it  removes  one  token  from  each 

/  input  place  and  creates  a  new  token  in  each  of  the  output  places  of  that 

!  transition.  One  can  envision  a  sequence  of  firings  in  the  Petri  Net  of 

|  Figure  2.1:  Let  the  initial  marking  consist  of  a  token  in  the  first  (left 
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most)  place.  Then  the  first  transition  is  enabled  and  it  fires.  The  token  in 
the  first  place  is  removed  and  a  token  appears  in  the  second  place.  Now  the 
second  transition  is  enabled:  it  fires  and  the  token  is  removed  from  the 
second  place;  a  new  one  appears  in  the  third  place,  and  so  on.  The  execution 
halts  when  the  fourth  transition  fires  and  a  token  appears  on  the  fifth  place. 

(•)  O  ^  e>o  o — — KD 


Figure  2.1.  A  Simple  Petri  Net 

A  transition  may  have  more  than  one  output  places.  When  it  fires,  a 
token  is  generated  in  each  output  place.  However,  to  model  decision  making, 
it  is  convenient  to  introduce  a  special  transition,  a  decision  switch,  in 
which  the  output  places  represent  alternatives.  When  the  decision  switch 
fires,  a  token  is  generated  in  only  one  of  the  output  places.  A  decision  rule 
associated  with  this  special  transition  determines  the  place  in  which  the 
token  is  generated.  The  rule  can  be  deterministic  or  stochastic;  it  can  be 
independent  of  the  attributes  of  the  tokens  in  the  input  places  or  it  may 
depend  on  them. 

A  subnet  of  a  Petri  Net  PN  is  a  Petri  Net  PNS  with  places  P3  that  are  a 
subset  of  the  places  P  of  the  original  net  and  transitions  Ts  that  are  a 
subset  of  the  transitions  T  of  the  original  net .  The  input  and  output 
mappings,  Is  and  0S,  are  restricted  to  the  arcs  between  the  subsets  Tg  and  Ps. 
The  use  of  subnets  simplifies  the  graphical  representation  of  complex 
organizations  and  allows  the  depiction  of  the  decisionmaker  model  at  a  level 
of  detail  appropriate  to  the  problem  being  solved. 

2.2  INFORMATION  THEORY 

Information  theory  was  first  developed  as  an  application  in  communication 
theory  [7].  But.  as  Khinchin  [8]  showed,  it  is  also  a  valid  mathematical 
theory  in  its  own  right,  and  it  is  useful  for  applications  in  many 
disciplines,  including  the  modeling  of  simple  human  decisionmaking  processes 
and  the  analysis  of  information-processing  systems. 

There  are  two  quantities  of  primary  interest  in  information  theory.  The 
first  of  these  is  entropy:  given  a  variable  x,  which  is  an  element  of  the 
alphabet  X,  and  occurs  with  probability  p(x),  the  entropy  of  x,  H(x),  is 
defined  to  be 

H(x)  ■  “  ^  p(x)  log  p(x)  (2.1) 
x 

and  is  measured  in  bits  when  the  base  of  the  algorithm  is  two.  The  other 
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quantity  of  interest  is  average  mutual  information  or  transmission:  given  two 
variables  x  and  y,  elements  of  the  alphabets  X  and  Y.  and  given  p(x),  p(y), 
and  p(xly)  (the  conditional  probability  of  x,  given  the  value  of  y)  ,  the 
transmission  between  x  and  y,  T(x:y)  is  defined  to  be 


T(x: y)  s  H(x)  -  H  (x) 


(2.2) 


where 

H  (x)  5  ~  5  P(y)  5  p(xly)  log  p(xly)  (2.3) 

y  tm  L 

y  x 

is  the  conditional  uncertainty  in  the  variable  x,  given  full  knowledge  of  the 
value  of  the  variable  y. 

McGill  [9]  generalized  this  basic  two-variable  input-output  theory  to  N 
dimensions  by  extending  Eq.  (2.2): 

N 

T(Xi  :xj  :  •  •  •  :Xjj)  5  ^  HUJ  -  H(xi>xj  , . . .  ,xN>  (2.4) 

i=l 


For  the  modeling  of  memory  and  of  sequential  inputs  which  are  dependent 
on  each  other,  the  use  of  the  entropy  rate,  H(x) ,  which  describes  the  average 
entropy  of  x  per  unit  time,  is  appropriate: 


H(x)  *  lim  —  H[x(t),  x(t+l) , . . . ,x(t+m-l)) 

\  m 
m->  ® 


(2.5) 


The  transmission  rate,  T(x:y),  is  defined  exactly  like  transmission,  but  using 
entropy  rate  in  the  definition  rather  than  entropy. 

The  Partition  Law  of  Information  [10]  is  defined  for  a  system  with  N-l 
internal  variables,  through  wN_x,  and  an  output  variable,  y,  also  called 
Wty.  The  law  states 


t 


ft 


and  is  easily  derived  using  information  theoretic  identities.  The  left-hand 
side  of  Eq.  (2.6)  refers  to  the  total  activity  of  the  system,  also  designated 
by  G.  Each  of  the  quantities  on  the  right-nano  side  has  its  own 
interpretation.  The  first  term,  T(x:y),  is  called  throughput  and  is 
designated  Gt.  It  measures  the  amount  by  which  the  output  of  the  system  is 
related  to  the  input.  The  second  quantity. 


Vx:vw,. 


,WN-l>  =  T(x:w1'wa« 


,wnl.y)  -  T ( x : y ) 


(2.7) 


is  called  blockage  and  is  designated  G^ .  Blockage  may  be  thought  of  as  the 
amount  of  information  in  the  input  to  the  system  that  is  not  included  in  the 
output.  The  third  term,  T(wx  :wa : . . . : w^_a :y)  is  called  coordination  and  is 
designated  GQ.  It  is  the  N-dimensional  transmission  of  the  system,  i.e.,  the 
amount  by  which  all  of  the  internal  variables  in  the  system  constrain  each 
other.  The  last  term,  Hx(w1,wa, . . . ,w^_lfy) ,  designated  by  Gn  represents  the 
uncertainty  that  remains  in  the  system  variables  when  the  input  is  completely 
known.  This  noise  should  not  be  construed  to  be  necessarily  undesirable,  as 
it  is  in  communication  theory:  it  may  also  be  thought  of  as  internally- 
generated  information  supplied  by  the  system  to  supplement  the  input  and 
facilitate  the  decisionmaking  process.  The  partition  law  may  be  abbreviated: 


G  =  G  +  G  +  G  +  G 
t  b  c  n 


(2.8) 


A  statement  completely  analogous  to  (2.8)  can  be  made  about  information 
rates  by  substituting  entropy  rate  and  transmission  rates  in  Eq.  (2.6). 

2.3  TASK  MODEL 

The  organization,  interacts  with  its  environment;  it  receives  signals  or 
messages  in  various  forms  that  contain  information  relevant  to  the 
organization's  tasks.  These  messages  must  be  identified,  analyzed,  and 
transmitted  to  their  appropriate  destinations  within  the  organization.  From 
this  perspective,  the  organisation  acts  as  an  information  user. 


Let  the  organization  receive  data  from  one  or  more  sources  (N')  external 
to  it.  Every  tq  units  of  time  on  the  average,  each  source  n  generates 
symbols,  signals,  or  messages  x^  from  its  associated  alphabet  XQ,  with 
probability  pni,  i.e.,  ~ 


P<V  lt„l)  !  xnl  8  Xn 


(2.9) 
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5  Pni  =  1  '*  n  =  1'2'***'N'  (2.10) 

i=l 

where  yn  is  the  dimension  of  xn.  Therefore,  l/tn  is  the  mean  frequency  of 
symbol  generation  from  source  n. 

The  organization's  task  is  defined  as  the  processing  of  the  input  symbols 
xn  to  produce  output  symbols.  This  definition  implies  that  the  organization 
designer  knows  a  priori  the  set  of  desired  responses  Y  and,  furthermore,  has  a 
function  or  table  L(xn)  that  associates  a  desired  response  or  a  set  of  desired 
responses,  elements  of  Y,  to  each  input  xn^  e  XR. 

It  is  assumed  that  a  specific  complex  task  that  must  be  performed  can  be 
modeled  by  N’  sources  of  data.  Rather  than  considering  these  sources 
separately,  one  supersource,  composed  of  these  N'  sources,  is  created.  The 
input  symbol  x'  may  be  represented  by  an  N '-dimensional  vector  with  each 
source  corresponding  to  a  component  of  this  vector,  i.e., 

x'  «  (x  .x  . . . .  ,x„ ,)  ;  x'  e  X  2.11) 

To  determine  the  probability  that  symbol  xl  is  generated,  the 
independence  between  components  must  be  considered.  if  all  components  are 
mutually  independent,  then  pj  is  the  product  of  the  probabilities  that  each 
component  of  Xj  takes  on  its  respective  value  from  its  associated  alphabet. 
If  two  or  more J components  are  probabilistically  dependent  on  each  other,  but 
as  a  group  are  mutually  independent  from  all  other  components  of  the 


N' 


input  vector,  then  these  dependent  components  can  be  treated  as  one 
supercomponent,  with  a  new  alphabet.  Then  a  new  input  vector,  x,  is  defined, 
composed  of  the  mutually  independent  components  and  these  super-components. 

Thi3  model  of  the  sources  implies  synchronization  between  the  generation 
of  the  individual  source  elements  so  that  they  may,  in  fact,  be  treated  as  one 
input  symbol.  Specifically,  it  is  assumed  that  the  mean  interarrival  time  xn 
for  each  component  is  equal  to  t.  It  is  also  assumed  that  the  generation  of  a 
particular  input  vector,  x,,  is  independent  of  the  symbols  generated  prior  to 
or  after  it. 

The  last  assumption  can  be  weakened,  if  the  source  is  a  discrete 
stationary  ergodic  one  with  constant  interarrival  time  x  that  could  be 
approximated  by  a  Markov  source.  Then  the  information  theoretic  framework  can 
be  retained  [11], 
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The  vector  output  of  the  source  is  partitioned  into  groups  of  components 
that  are  assigned  to  different  organization  members.  The  j-th  partition  is 
denoted  by  x^  and  is  derived  from  the  corresponding  partition  matrix  which 
has  dimension  nj  x  N  and  rank  n  j ,  i.e.. 


x 


J 


(2.13) 


Each  column  of  rr^  has  at  most  one  non-zero  element.  The  resulting  vectors  x^ 
may  have  some,  all,  or  no  components  in  common. 

The  set  of  partitioning  matrices  {jt1  ,tt* , . . .  ,nn}  shown  in  Figure  2.2 
specify  the  components  of  the  input  vector  received  by  each  member  of  the 
subset  of  decisionmakers  that  interact  directly  with  the  organization's 
environment.  These  assignments  can  be  time  invariant  or  time  varying.  In  the 
latter  case,  the  partition  matrix  can  be  expressed  as 


n^Ct)  = 


for  t  e  {T} 
for  t  f  {T} 


(2.14) 


The  times  {T}  at  which  a  decisionmaker  receives  inputs  for  processing  can  be 
obtained  either  through  a  deterministic  (e.g.,  periodic)  or  a  stochastic  rule. 
The  question  of  how  to  select  the  set  of  partition  matrices,  i.e.,  design  the 
information  structure  between  the  environment  and  the  organization,  has  been 
addressed  by  Stabile  [12,13]. 


Figure  2.2.  Information  Structures  for  Organizations 
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2.4  THE  DECISIONMAKER  MODEL 

The  basic  model  of  the  memoryless  decisionmaker  with  bounded  rationality 
is  based  on  the  hypothesis  of  F.  C.  Donders  [14]  that  information  processing 
is  done  in  stages.  Specifically,  it  is  assumed  that  the  two  stages  are  (a) 
situation  assessment  (SA) ,  and  (b)  response  selection  (RS) ,  which  correspond 
to  March  and  Simon's  [15]  two  stage  process  of  discovery  and  selection.  The 
structure  of  this  model,  shown  in  Figure  2.3,  has  been  extended  to  include 
interactions  with  other  organization  members,  as  well  as  memory.  The  extended 
model  is  shown  in  Figure  2.4. 


Figure  2.3  Two-Stage  Model 
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processed  together  in  the  situation  assessment  (SA)  stage.  The  SA  stage 
contains  algorithms  that  process  the  incoming  signals  to  obtain  the  assessed 
situation  z.  The  SA  stage  may  access  the  memory  or  internal  data  base  to 
obtain  a  set  of  values  dQ.  The  assessed  situation  z  may  be  shared  with  other 
organization  members;  concurrently,  the  DM  may  receive  the  supplementary 
situation  assessment  z'  from  other  parts  of  the  organization;  the  two  sets  z 
and  z'  are  combined  in  the  information  fusion  (IF)  processing  stage  to  obtain 
z.  Some  of  the  data  (dj)  from  the  IF  process  may  be  stored  in  memory. 

The  possibility  of  receiving  commands  from  other  organization  members  is 
modeled  by  the  variable  v'.  A  command  interpretation  (Cl)  stage  of  processing 
is  necessary  to  combine  the  situation  assessment  2  and  v'  to  arrive  at  the 
choice  V  of  the  appropriate  strategy  to  use  in  the  response  selection  (RS) 
stage .  The  RS  stage  contains  algorithms  that  produce  outputs  y  in  response  to 
the  situation  assessment  z  and  the  command  inputs.  The  RS  stage  may  access 
data  from,  or  store  data  in  memory  [11] . 

In  this  report,  only  the  memoryless  case  is  considered.  Consequently, 
the  general  model  reduces  to  the  one  shown  Figure  2.5,  where  the  Petri  Net 
formalism  has  been  used . 


Figure  2.5.  The  Memoryless  Interacting  Decisionmaker  Model 


A  more  detailed  description  of  the  model  is  obtained,  if  the  internal 
structure  of  the  SA  and  RS  stages  is  considered.  The  situation  assessment 
stage  consists  of  a  set  of  U  algorithms  (deterministic  or  not)  that  are 
capable  of  producing  some  situation  assessment  z.  The  choice  of  algorithms  is 
achieved  through  specification  of  the  internal  variable  u  in  accordance  with 
the  situation  assessment  strategy  p(u),  or  p(ulz),  if  a  decision  aid  (e.g.,  a 
preprocessor)  is  present.  A  second  internal  decision  is  the  selection  of  the 
algorithm  in  the  RS  stage  according  to  the  response  selection  strategy 
p(7|z,v') .  The  two  strategies,  when  taken  together,  constitute  the  internal 
decision  strategy  of  the  decisionmaker. 

The  subnets  representing  the  SA  and  the  RS  stages  are  shown  in  Figure 
2.6.  Note  the  presence  of  decision  switches  in  place  of  the  regular 
transitions  to  indicate  that  only  one  of  the  output  places  can  receive  a  token 
at  each  firing. 
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Figure  2.6.  The  SA  and  the  RS  Subnets 

2.5  WORKLOAD 

The  analytical  framework  presented  in  Section  2.2,  when  applied  to  the 
single  interacting  decisionmaker  with  deterministic  algorithms  in  the  SA  and 
RS  stages,  yields  the  four  aggregate  quantities  that  characterize  the 
information  processing  and  decisionmaking  activity  within  the  DM: 

Throughput: 

Gt  =  T(x,z\v':z,y)  (2.15) 

Blockage: 

G.  -  H(x ,z ' ,v* )  -  G.  (2.16) 

o  t 

Internally  generated  inforaation: 

G  =  H(u)  -  H_(v)  (2.17) 

n  z 

Coordination: 

U 

G  =  ^  p,g*(p(x))  +  a  H(p  )  +  H(z)  +  gIF  (p(z,z'))  +  gCI  (p(Z,v ') 

C  i.  1  C  11  c  c 

i=l 

V 

+  \  Pjgg(p(Zl7  -  j))  +  ■  Hfpj)  +  H(y)  +  H(z)  +  H(Z)  +  H(Z,V) 
j=l 

+  T  (x* :z')  +  T  (x* ,z':v')  (2.18) 

z  z 
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The  expression  for  Gn  shows  that  it  depends  on  the  two  internal 
strategies  p(u)  and  p(vlz)  even  though  a  command  input  may  exist.  This 
implies  that  the  command  input  v'  modifies  the  DM's  internal  decision  after 
p(v|z)  has  been  determined. 

In  the  expressions  defining  the  system  coordination,  pi  is  the 
probability  that  algorithm  f^  has  been  selected  for  processing  the  input  x  and 
p,  is  the  probability  that  algorithm  h,  has  been  selected,  i.e.,  u  =  i  and 

I-W  ,  »T1  1 _  _ _ _  k  k  i  >  .  A  **.  U  -  4*  i _ 


The  quantities 


represent 


internal  coordinations 


corresponding  algorithms  and  depend  on  the  probability  distribution  of  their 
respective  inputs;  the  quantities  a^,  aj  are  the  number  of  internal  variables 
of  the  algorithms  fj^  and  hj,  respectively.  Finally,  the  quantity  H  is  the 
entropy  of  a  binary  randoni  variable  that  takes  one  of  its  two  values  with 
probability  p. 


H(p)  =  -  p  log2  p  -  (1  -  p)log2(l-p) 


(2.19) 


Equations  (2.15)  tb  (2.18) . determine  the  total  activity  G  of  the  decisionmaker 
according  to  the  partition  law  of  information,  Eq.  (2.6).  The  activity  G  can 
be  evaluated  alternatively  as  the  sum  of  the  marginal  uncertainties  of  each 
system  variable.  For  any  given  internal  decision  strategy.  G  and  its  component 
parts  can  be  computed . 

Since  the  quantity  G  may  be  interpreted  as  the  total  information 
processing  activity  of  the  system,  it  can  serve  as  a  surrogate  for  the 
workload  of  the  organization  member  in  carrying  out  his  decisionmaking  task . 

The  qualitative  notion  that  the  rationality  of  a  human  decisionmaker  is 
not  perfect,  but  is  bounded  [16],  has  been  modeled  as  a  constraint  on  the 
total  activity  G.  The  specific  form  for  the  constraint  has  been  suggested  by 
the  empirical  relation 


c  +  c  G. 

i  *  t 


where  t  is  the  average  reaction  time,  i.e.,  the  time  between  the  arrival  of 
the  input  and  the  generation  of  an  output  y.  It  is  assumed  that  the 
decisionmaker  must  process  his  inputs  at  a  rate  that  is  at  least  equal  to  the 
rate  with  which  inputs  arrive.  The  latter  has  been  modeled  by  x,  the  mean 
symbol  interarrival  time; 


The  modeling  assumptions  in  this  paper  are  that 


c 

X 


c 
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+  G 
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+  G 
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and  that  c,  does  not  depend  on  p(x)  .  Then,  the  bounded  rationality  constraint 
takes  the  form 


G  =  G.  +  G.  +  G 
t  b  n 


G 

c 


(2.20) 


where  F  can  be  considered  as  a  rate  of  total  activity  and  is  measured  in  bits 
per  second.  Inequality  (2.20)  represents  a  mathematical  expression  of  only 
one  aspect  of  bounded  rationality.  Many  other  formulations  are  possible. 

Weakening  the  assumption  that  the  algorithms  are  deterministic  changes 
the  numerical  values  of  GQ  and  of  the  coordination  term  G0  [17]  .  If  memory  is 
present  in  the  model,  then  additional  terms  appear  in  the  expressions  for  the 
coordination  rate  and  for  the  internally  generated  information  rate  [11]. 

2.6  MEASURES  OF  PERFORMANCE 

As  stated  in  Section  2.3,  it  is  assumed  that  the  designer  knows  a  priori 
the  set  of  desired  responses  Y  to  the  input  set  X .  One  measure  of  performance 
(MOP)  of  the  organization  that  reflects  the  degree  to  which  the  actual 
response  matches  the  desired  response  can  be  computed  as  shown  in  Figure  2.7. 


Figure  2.7.  Performance  Evaluation  of  an  Organization 


The  decisionmaker's  actual  response  y  can  be  compared  to  the  desired 
response  y'  and  a  cost  is  assigned  using  the  cost  function  d(y.y').  If  this 
function  is  a  binary  one,  i.e., 

|  0  if  y  =  y' 

d(y,y')  =  )  (2.21) 

(  1  if  y  ^  y’ 

then  the  expected  value  of  this  cost  denotes  the  probability  that  the  wrong 
decision  is  made,  i.e.,  it  is  the  probability  of  error. 

In  general,  however,  there  is  a  cost  c^j  associated  with  selecting  e 
Y  when  the  desired  response  is  y.  e  Y': 

J 

Cij  =  d(yi'yj)  (2.22) 

so  that 

J  -  }  1  “i/O'ih*  <2-23’ 

J  i 
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where  y,  is  the  desired  response  to  task  x j .  This  measure  of  performance  can 
be  Interpreted  as  a  measure  of  the  accuracy  of  the  response,  to  the  extent 
that  a  cost  is  associated  with  the  degree  with  which  the  actual  decision 
deviates  from  the  desired  one. 

This  class  of  performance  measures,  described  generically  by  (2.23),  is 
not  the  only  one  that  has  been  considered.  In  related  work  [18],  measures  of 
performance  that  address  time  have  been  modeled  sued  analyzed. 

2.7  PERFORMANCE-WORKLOAD  LOCUS 

A  useful  way  of  describing  the  properties  of  the  decisionmaker  model, 
which  is  generalizable  to  the  properties  of  am  organization,  is  through  the 
performance  workload  locus.  In  the  case  of  a  single  performance  measure,  the 
accuracy  measure  J,  and  a  single  decisionmaker  with  workload  G,  a  two 
dimensional  space  is  defined  with  ordinate  J  and  abscissa  G.  The  locus  is 
constructed  by  considering  the  functional  dependence  of  J  and  G  on  the 
internal  decision  strategies  of  the  single  decisionmaker. 

Let  an  internal  strategy  for  a  given  decisionmaker  be  defined  as  pure,  if 
both  the  situation  assessment  strategy  p(u)  and  the  response  selection 
strategy  p(vlz)  are  pure,  i.e.,  an  algorithm  f,  is  selected  with  probability 
one  and  an  algorithm  hi  is  selected  also  with  probability  one  when  the 
situation  is  assessed  as  'neing  Z: 


=  (p(u=i)  =  1 


p (v=j I z=Z)  =  1} 


(2.24) 


D 


k 


for  some  i,  some  j,  and  for  each  Z  element  of  the  alphabet  Z.  There  are  n 
possible  pure  internal  strategies. 


n  =  U'V*1  (2.25) 

where  U  is  the  number  of  f  algorithms  in  the  SA  stage,  V  the  number  of  h 
algorithm  in  the  RS  stage  and  M  the  dimension  of  the  set  Z.  All  other 
internal  strategies  are  mixed  [1]  and  are  obtained  as  convex  combinations  of 
pure  strategies: 


D(Pk) 


n 


■  l  pkDk 

k=l 


(2.26) 


where  the  weighting  coefficients  are  probabilities. 

Corresponding  to  each  D(pk)  is  a  point  in  the  simplex 
n 

^  Pk  =  1#  Pk  -  0  y  k  (2.27) 

k=l 


The  possible  strategies  for  an  individual  DM  are  elements  of  a  closed  convex 
polyhedron  of  dimension  n-1  whose  vertices  are  the  unit  vectors  corresponding 
to  pure  strategies. 

The  total  activity  G,  the  surrogate  for  the  cognitive  workload,  is  a 
convex  function  of  the  decision  strategy,  i.e.. 


n 

G  (D(pk))  >  ^  pk  Gk  (2.28) 

k=l 

where  Gk  is  the  workload  that  results  when  the  pure  strategy  Dk,  given  by  Eq. 
(2.24) ,  is  used . 

The  accuracy  measure  J  can  be  related  to  the  decision  strategies  in  a 
similar  manner.  Corresponding  to  each  pure  strategy  Dk  is  a  value  of  the 
performance  measure,  denoted  by  Jk.  Since  each  strategy  is  a  convex 
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combination  of  pure  strategies,  the  value  of  J  for  an  arbitrary  D(p^)  is  given 
as  a  convex  combination  of  the  values  of  J^,  i.e.. 


n 

J(D(p  )£  =  ^  P  kJ  k  (2.29) 

k=l 


The  two  expressions,  (2.28)  and  (2.29)  can  be  used  now  to  characterize  the 
locus  of  points  in  the  (J,G)  space  that  describe  the  decisionmaker. 

Example :  Consider  first  the  case  of  two  pure  strategies,  D2  and  D2  .  This 

would  correspond  to  the  case  where  the  decisionmaker  can  choose  only  between 
two  different  algorithms  f  in  the  SA  stage,  as  shown  in  Figure  2.8.  The 
strategy  space  for  this  case  can  be  parametrized  as  follows:  Any  strategy,  D, 
can  be  expressed  as 

D  =  Dx  +  Pj  D2  (2.30) 


where  p2  +  p2  =1 

in  accordance  with  (2.26)  and  (2.27).  Let 
p  =1-6  and  p  =6 

i  i 


and  let 


0  <  6  <  1 


Then,  (2.30)  can  be  rewritten  as 


D  =  (1-6)  D  +  6D 

i  * 


(2.31) 


Figure  2.8.  The  Petri  Net  for  the  Example 


The  strategy  space  can  be  described  by  the  parameter  6:  it  is  the  line 
segment  [0,1],  as  shown  in  Figure  2.9,  with  the  point  0  corresponding  to  pure 
strategy  Dx,  point  1  to  pure  strategy  D,,  and  all  points  in  between  to  all  the 
mixed  strategies. 


0  1 

Figure  2.9.  Strategy  Space  for  Example 


Then,  it  follows  from  (2.28)  and  (2.29)  that 

G(D(pk))  =  G(D(6) )  >  (1-6) G^  +  6  G%  (2.32) 

and 

J(D(pk))  =  J(D (5) )  =  (1-6) +  6  J#  33 

Equations  (2.32)  and  (2.33)  are  parametric  in  6  and  result  in  the  locus  shown 
in  Figure  2.10.  The  relative  position  of  the  end  points  (J^Gj)  and  (Jj.G,) 
is  problem  specific;  it  is  not  true  that  smaller  workload  leads  to  worse 
performance,  as  Figure  2.10  indicates. 


WORKLOAD 


Figure  2.10.  Performance-Workload  Locus  for  Example 


In  the  general  case,  there  are  n  pure  strategies,  as  given  by  Eq .  (2.25). 
Then,  the  P-W  locus  is  constructed  as  follows: 

First,  the  values  of  for  the  n  pure  strategies  are  determined. 
This  corresponds  to  evaluating  the  performance  and  the  workload  for  the  values 
of  p^,  Eq.  (2.27),  that  correspond  to  the  vertices  of  the  strategy  space.  The 
result  is  a  set  of  n  points  in  the  two-dimensional  P-W  space. 

Then,  the  binary  variations  between  each  possible  pair  of  pure  strategies 
are  considered.  This  corresponds  to  the  mapping  of  the  edges  of  the  strategy 
space.  For  example,  consider  pure  strategies  and  then 

D  =  (l-6)Di  +  6Dk 

for  all  combinations  (i,k)  where  i=l,...,n  and  k=l,...,n  and  for  which  ijtk. 
By  varying  6  from  0  to  1,  the  loci  (Jil{(6)  ,  Gik(6))  are  obtained.  These  are 
convex  lines  joining  the  two  boundary  points,  as  shown  in  Figure  2.10.  These 
binary  loci  are  quite  useful,  since  they  define  the  minimum  workload  locus  for 
any  feasible  value  of  J. 

The  third  step  consists  of  considering,  successively,  the  binary 
variation  between  all  possible  binary  strategies  until  all  mixed  strategies 
are  accounted  for.  The  result  is  a  locus  such  as  the  one  shown  in  Figure  2.11 
for  the  case  when  there  are  three  pure  strategies.  The  corresponding  strategy 
space,  for  this  case,  is  shown  in  Figure  2.12. 
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Figure  2.11.  Performance-Workload  Locus  for  the  Case  of  Three  Pure  Strategies 
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Figure  2.12.  Strategy  Space  for  the  Case  of  Three  Pure  Strategies 

Thus,  the  decisionmaker  model  can  be  considered  as  a  system  that  maps  the 
strategy  locus,  the  simplex  defined  by  Eq.  (2.27),  into  the  Performance- 
Workload  Locus  (J,G).  Any  change  in  the  algorithms  f  or  h,  or  the  functions 
in  IF  and  Cl,  or  the  input  x  will  affect  the  mapping. 


Any  specific  Instantiation  of  the  decisionmaker  model  can  be  analyzed  by 
considering  the  strategy  space  and  the  corresponding  P-W  locus.  The  detailed 
procedure  for  doing  both  a  qualitative  and  a  quantitative  evaluation  will  be 
presented  in  the  next  chapter. 


3.  THE  DESIGNER'S  AND  THE  DECISIONMAKERS 'S  POINTS  OF  VIEW* 

When  the  tasks  which  have  to  be  performed  exceed  the  capabilities  of  a 
single  person,  organizations  are  formed,  consisting  of  individuals  interacting 
with  each  other  in  specific  ways.  The  most  important  characteristic  of  the 
organizations  considered  here  is  that  their  task  involves  information 
processing  and  decisionmaking. 

In  earlier  work  [1-3],  several  simplifying  assymptions  has  been  made  in 
the  design  and  analysis  of  decisionmaking  organizations:  (a)  designers  and 
decisionmakers  have  identical  knowledge  of  the  tasks'  uncertainty,  i.e.,  the 
probability  distribution  of  the  tasks  and  (b)  identical  perception  of  the 
value  of  each  task.  These  assumptions  are  very  restrictive  and  often 
unjustifiable.  In  general,  it  is  very  difficult  to  assess  the  probability 
distribution  of  the  tasks;  it  is  also  unlikely  that  the  designer  and  the 
decisionmakers  have  the  same  perception  of  the  tasks'  probability 
distribution.  In  this  chapter,  this  assumption  is  relaxed. 

It  is  assumed  that  the  designer  knows  the  tasks'  real  probability 
distribution,  while  the  decisionmaker's  peception  of  this  distribution  is 
different.  The  second  assumption,  that  all  tasks  are  of  equal  value,  is  also 
improbable;  usually,  different  tasks  have  different  utilities,  i.e.,  different 
weights  are  assigned  to  them  by  the  designer.  Therefore,  the  second 
assumption  must  be  weakened,  so  that  each  task  can  be  weighted  differently. 
In  order  to  pose  these  problems  properly,  two  additional  assumptions  must  be 
introduced:  (a)  there  is  no  communication  between  the  designer  and  the 
decisionmakers  and  (b)  the  designer  knows  the  tasks'  uncertainty  as  perceived 
by  each  decisionmaker  in  the  organization,  as  well  as  the  relative  weights 
assigned  by  them  to  each  task. 

3.1  THE  ORGANIZATION  MODEL 

The  task  consists  of  receiving  data  (signals) ,  processing  that  data,  and 
producing  an  output  in  the  form  of  actions  or  signals.  The  input  data  can 
originate  from  a  single  source  or  from  many  different  sources  as  described  in 
the  previous  chapter.  The  data  may  be  a  single  element  or  a  set  of  elements. 
In  general,  it  is  modeled  as  a  vector,  x,  generated  by  a  single  source.  This 
vector  signal  is  partitioned  by  partitioning  matrices  n  and  allocated  to  the 
appropriate  decisionmakers .  A  task  can  be  specified  fully  by  its  finite 
scheme,  which  consist  of  the  task's  alphabet  and  its  probability  distribution, 
i  .e.. 


X  =  (X, 


(3.1) 


•This  section  is  based  on  the  work  of  M.  M.  Tomovic  [18] ;  the  computational 
results  have  been  revised  recently  by  J.  Azzola. 


The  basic  model  of  the  memoryless  decisionmaker  introduced  in  the 
previous  chapter  will  be  used.  It  is  assumed  that  the  overall  task  x  is 
partitioned  by  the  matrix  nn  and  that  only  the  appropriate  elements  of  x  are 
allocated  to  the  n-th  decisionmaker.  The  decisionmaker  processes  the  data 
using  the  algorithms  in  the  SA  stage  in  order  to  assess  the  situation.  The 
assessed  situation,  Zn,  is  then  processed  in  the  RS  stage,  where  the  decision 
of  an  appropriate  action  or  response,  yn,  is  made.  Which  ones  of  the  SA  and 
RS  algorithms  a  decisionmaker  will  use  depends  on  his  choice  of  internal 
decision  strategy,  d£.  For  the  situation  assessment  stage,  it  is  assumed  that 
the  strategy  un  is  independent  of  the  input  xn,  whereas  in  the  response 
selection  stage  vn  depends  on  the  value  of  the  assessed  situation  zn.  THe 
assumption  that  the  choice  of  un  is  not  dependent  on  xn  has  been  relaxed  by 
Chyen  and  Levis  [17]. 

In  order  to  make  communication  and  interaction  between  decisionmakers 
possible,  the  information  fusion  (IF),  and  the  command  interpretation  (Cl) 
elements  have  been  included.  The  IF  process  allows  sharing  of  the  information 
on  the  state  of  environment  between  decisionmakers.  This  functional  element 
associates  information  on  the  assessed  situation  obtained  by  the  n-th 
decisionmaker,  zn,  and  the  corresponding  information  sent  to  him  from  the  rest 
of  organization,  zon,  and  gives  the  cumulative  updated  information  on  the 
state  of  environment  zn.  It  is  also  possible  for  the  n-th  decisionmaker  to 
communicate  his  knowledge  on  the  state  of  environment  zn0  to  other  members  of 
the  organization,  who  accept  and  fuse  that  information  with  their  own  in  the 
corresponding  IF  stage.  Commands  von  received  by  the  n-th  decisionmaker  from 
the  rest  of  the  organization  can  modify  or  even  override  his  own  decision  vn. 

3.2  WORKLOAD  AND  PERFORMANCE 

The  concepts  of  decision  stategies,  workload,  and  performance  introduced 
in  Chapter  2  for  the  single  decisionmaker  need  to  be  generalized  to  apply  to 
organizations  consisting  of  a  number  of  decisionmakers  operating  as  a  team  of 
accomplish  the  given  set  of  tasks.  As  before,  the  two  fundamental  quantities 
are  the  workload  or  the  activity  level  of  each  individual  DM  and  the 
performance  index  of  the  organization.  Both  of  them  are  functions  of  the 
input,  the  decisionmaker's  internal  structures,  the  organization's  protocol  or 
standard  operating  procedures  and  the  decision  strategies.  For  a  specified 
input,  a  protocol  and  am  internal  structure  (set  of  algorithms)  both  the 
performance  and  the  workload  depend  parametrically  on  the  organizational 
behavioral  strategy  which  is  defined  as 

A  =  (Dl(p1)  . DV)  -  }  ••  J  V  kN  Pk  '*•  PKN  (3,2) 

kl  ^ 

where  DnXpn)  Is  the  n-th  decisionmaker 's  mixed  decision  strategy,  and 
, . . . ,kN  denotes  a  pure  strategy  of  the  organization  (see  Levis  and 
Boettcher  [2]). 


The  surrogate  workload  for  each  decisionmaker  is  defined  as  the  activity 
necessary  to  reduce  uncertainty  and  arrive  at  a  decision.  For  each  DM  it  is 
defined  as 

Gn  =  Gn(A)  =  -  ^  J  p (wjn)  loS  P<wjn>  (3.3) 

i  j 

where  w^n  is  the  j-th  value  of  the  i-th  internal  variable  of  the  n-th 
decisionmaker .  When  the  tasks'  finite  scheme  X,  organizational  structure 
(algorithms  and  protocols)  and  organization's  behavioral  strategy  A,  Eq. 
(3.2),  are  specified,  it  is  then  possible  to  evaluate  the  activity  of  each 
decisionmaker  within  the  organization. 

In  has  been  shown  in  Chapter  2  that  the  total  activity  G  is  a  convex 
function  of  the  decision  strategy  A  in  the  sense  that 

G(A)  >  J  ...  J  G(Ak1...kN)  pkx"*  pkN  (3,4) 


where  G(Aki  kN)  is  the  workload  corresponding  to  the  pure  strategy. 

The  bounded  rationality  of  each  decisionmaker  is  modeled  as  a  constraint 
on  his  total  activity 

Gn(A)  <  Fnx  (3.5) 

where  Fn  is  the  maximum  rate  at  which  the  n-th  decisionmaker  can  process 
information  and  t  is  the  mean  interarrival  time  of  tasks. 

The  organization's  performance  is  defined  as 

J  =  Eld(y,y’ ) )  =  ^  pU^  ^  d(yj,yj)  p(yjlxi)  (3.6) 

i  J 

where  y,  is  the  actual  output  of  the  organization  as  a  whole  in  response  to 
the  input  xA  and  where  yj  is  the  designer.  As  in  the  case  of  a  single 
decisionmaker,  the  comparison  function  d(y,y')  can  take  any  form  appropriate 
to  the  particular  problem;  in  its  simplest  form  it  is  defined  as 


d(y.y')  = 


-C  : 


y  =  y’ 


y  t  y' 


(3.7) 


In  that  case,  the  performance  index  is  reduced  to  the  probability  of  producing 
an  incorrect  response,  i.e.,  the  probability  of  making  an  error. 

The  performance  measure  has  meaning  only  for  the  organization  as  a  whole 
and  can  be  expressed  as 


KA)  «  }  ...  \ 


V...kN  Pk1  pkN 


(3.8) 


to  show  its  dependence  on  the  choice  of  an  organizational  strategy.  The 
designer  is  the  one  who  assigns  the  value  of  the  performance  threshold  (J) 
which  the  decisionmaking  organization  has  to  meet,  i.e.. 


J  (A)  <  J 


(3.9) 


This  condition  determines  the  set  of  strategies  that  yield  satisficing 
performance . 

3.3  THE  PERFORMANCE-WORKLOAD  LOCUS 

For  each  decision  strategy,  the  organization's  performance  and  the 
workload  of  each  of  the  N  decisionmakers  can  be  computed  using  the  algorithms 
first  developed  by  Boettcher  [20],  revised  by  M.  Tomovic  [19]  and  then 
completely  revised  by  Andreadakis  [18]  as  part  of  the  Computer-Aided 
Evaluation  of  System  Architectures  (CEASAR)  workstation.  The  performance- 
workload  locus,  SQ,  is  defined  in  the  (N+l) -dimensional  space  S  as  the  set  of 
all  points  (J.G1, . . . ,GN)  that  correspond  to  the  set  of  all  admissible  decision 
strategies.  Let  I  be  the  set  of  all  admissible  strategies  of  the 
organization;  then 


I  -  l  ( f.t . f) 


(3.10) 


m 

Kl 
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where  £n  is  the  set  of  admissible  strategies  of  the  n-th  decisionmaker. 

The  bounded  rationality  contrainst  for  each  DM,  Eq.  (3.5)  is  expressed 
simply  as  a  bounding  hyperplane  in  the  performance-workload  space;  the 
satisficing  constraint,  _Eq.  (3.9),  is  also  a  bounding  hyperplane  that 
intersects  the  J  axis  at  J. 

The  resulting  performance-workload  locus  can  be  used  to  compare 


qualitatively  two  different  organizational  structures  or  designs  that  are  to 
perform  the  same  task.  It  can  also  be  used  to  compare  the  effectiveness  of 
the  same  organization  in  dealing  with  different  degrees  of  uncertainty,  i.e., 
for  different  probability  distributions  of  the  input  alphabet  X,  the  set  of 
tasks.  This  qualitative  analysis  is  based  on  how  well  the  locus  SQ  meets  the 
requirements  of  Eqs.  (3.5)  and  (3.9). 

A  quantitative  approach  to  the  comparison  and  effectiveness  analysis  of 
organizational  forms  is  based  on  comparisons  not  in  the  performance-worklaod 
space,  but  in  the  strategy  space.  Bounded  rationality  contraints,  Eq.  (3.5), 
and  performance  requirements  ,  Eq.  (3.9),  partition  the  space  of  organization 
strategies  into  a  set  of  strategies  that  lead  the  points  in  the  performance- 
workload  locus  that  satisfy  constraints  Eqs.  (3.5)  and  (3.9),  and  a  set  that 
does  not.  The  set  of  feasible  strategies  is  defined  as  follows: 

I'=  I  (A|J(A)  <  3.  G* (A)  1  FXt, - GN(A)  <  FNt)  (3.11) 

Then,  a  measure  of  effectiveness,  Q,  called  the  consistency  measure,  can  be 
defined  [3] : 

Q  =  V(7')/V(I)  '  (3.12) 

where  V  is  a  volume  in  the  N-dimensional  strategy  space.  Therefore,  Q  denotes 
the  fraction  of  all  strategies  that  are  feasible.  The  higher  it  is,  the  more 
consistent  the  decisions  of  the  organization  will  be  in  the  design 
specifications.  A  second  interpretation  of  the  measure  is  the  following:  if 
all  organizational  decision  strategies  are  equally  probable,  then  Q  is  the 
probability  that  the  organization  will  make  a  decision  that  satisfies  the 
individual  bounded  rationality  constraints  and  leads  to  satisficing 
performance ._  Hence,  Q  is  a  bounded  non-decreasing  function  of  the  performance 
threshold  J  and  individual  decisionmaker's  activity  threshold  Fnx, 
(n=l, . . . ,N)  . 

0  <  Q(J,F1t,  F*t,...,FNt)  <  1  (3.13) 

The  consistency  measure  Q  is  equal  to  zero,  if  there  is  no  decision  strategy 
which  meets  the  specifications  of  the  task.  Its  value  is  equal  to  unity,  if 
all  admissible  decision  strategies  satisfy  the  requirements.  It  is  evident 
that  the  higher  value  of  the  mutual  consistency  measure  Q,  the  better 
performance  can  be  expected  from  that  system.  Therefore,  the  designer  can 
compare  systems  with  respect  to  Q  and  select  the  one  with  the  highest  value  of 
Q  for  a  given  J  and  x. 

A  more  useful  representation  of  Q  is  in  terms  of  the  performance 
threshold  3  and  the  mean  interarrival  time  x.  The  resulting  expression 


V 


0  <  Q(J,x)  <  1 


(3.14) 


can  be  depicted  in  a  three-dimensional  space  regardless  of  the  size  of  the 
organization. 

3.4  TWO  POINTS  OF  VIEW 

The  basic  premise  in  earlier  work  is  that  the  designer  knows  the  finite 
scheme,  Eq.  (3.1),  and  the  performance  index  and  that  he  assumes  the 
decisionmakers  in  the  organization  will  have  the  same  knowledge.  On  that 
basis,  he  can  design  an  organizational  form  and  evaluate  it.  However,  the 
question  arises  as  to  how  robust  the  design  is,  i.e.,  how  sentitive  is  the 
value  of  Q  to  the  assumption  that  the  decisionmakers  have  indeed  the  same 
knowledge.  Suppose,  that  they  have  different  perceptions  of  the  tasks' 
uncertainty  and  they  assign  different  values  to  the  inidividual  tasks.  The 
inidivudal  decisionmakers  who  only  receive  partial  information  about  the 
organization's  tasks  may  have  a  different  perception  of  the  probability 
distribution  p(x)  .  Furthermore,  their  local  objectives  may  distort  the  values 
of  the  weights  assigned  to  each  task  by  the  designer  who  maintains  a  global 
perspective . 

The  designer  can  adopt  two  points  of  view  in  order  to  study  the 
robustness  of  his  design.  First,  he  can  assume  that  the  DMs  will  operate  on 
the  basis  of  his  perception  of  task  uncertainty,  p(x),  and  objective  function 
J.  Or,  he  can  assume  that  the  DMs  will  operate  on  the  basis  of  a  different 
perception  of  task  uncertainty,  e.g.,  q(x),  and  that  they  will  assign 
different  values,  c(x),  to  the  various  tasks  . 

These  two  points  of  view  lead  to  the  formualtion  of  four  problems  that 
the  designers  must  analyze. 

Problem  1 :  (Basic  Problem)  The  DMs  know  the  objective  probability 
distribution  of  the  tasks,  p(x),  and  the  weighting  coefficients  for  the 
various  tasks.  For  simplicity,  it  is  assumed  that  all  the  coefficients  c(x) 
are  equal  to  unity  for  the  base  case. 

Problem  2 :  (Task  Uncertainty)  The  DMs  have  their  own  perception  of  the 
probability  distribution  of  the  input,  q(x),  instead  of  p(x),  but  assign  the 
same  values  to  the  tasks  as  the  designer  (c(x)  =  1). 


Problem  3;  (Task  Value)  The  DMs  know  the  objective  probability  distribution 
to  the  tasks,  p(x),  but  assign  different  values  of  the  various  tasks,  i.e., 
their  c(x)  differs  from  the  designer's  (c(x)  £  1). 

Problem  4:  (General  Problem)  The  DMs  have  their  own  perception  of  the 

probability  distribution  of  the  input,  q(x),  and  the  value  of  the  tasks 
(c(x)jtl). 
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Problem  1  is  the  one  that  has  been  analyzed  in  detail  in  [1].[2],  and  [3] 
while  Problem  4  is  the  general  case:  Problems  1,  2,  and  3  are  really  special 
cases  of  4.  In  this  study.  Problem  2  is  analyzed  because  it  addresses  the 
complex  interrelationship  between  uncertainty  and  workload. 

It  is  possible  to  develop  the  analytical  expressions  for  the  workload  as 
a  function  of  p(x)  and  of  q(x)  .  It  is  also  possible  to  introduce  weighted 
entropy  [21],  and  derive  the  expressions  for  the  difference  in  the  workload  Gn 
due  to  the  difference  in  the  perception  of  the  task  uncertainty  by  the 
designer  and  the  decisionmakers.  However,  the  analytical  expressions  have  not 
yielded  any  particular  insight  to  the  problem.  Consequently,  the  sensitivity 
analysis  will  be  described  in  terms  of  an  example. 

3.5  TWO  THREE-PERSON  ORGANIZATIONS 

Two  three-person  organizations  have  been  used  to  illustrate  the  analysis 
and  comparison  of  alternative  organizational  forms.  This  simple  example  has 
been  based  on  the  problem  of  designing  the  organization  of  batteries  of 
surface-to-air  missiles. 

Let  the  trajectory  of  a  target  be  defined  by  an  ordered  pair  of  points 
located  in  a  rectangle  that  represents  a  two-dimensional  (flat)  sector  of 
airspace.  From  the  ordered  pair,  the  speed  and  direction  of  flight  of  the 
target  can  be  determined.  On  the  basis  of  that  information,  the  organization 
should  respond  by  firing  either  a  slow  or  a  fast  surface-to-air  missile  or  by 
not  firing  at  all.  It  is  assumed  that  the  size  of  that  sector,  the  frequency 
of  the  arrival  of  targets,  and  the  response  time  of  the  weapons  systems  are 
such  that  three  units  are  needed,  i.e.,  it  is  necessary  to  design  a  three- 
member  organization  to  accomplish  the  task.  Two  such  organizational 
structures  are  considered . 

The  first  structure.  Organization  A,  is  a  parallel  structure  with  lateral 
links,  and  is  defined  as  follows.  The  rectangular  sector  is  divided  into 
three  equal  subsectors  and  a  decisionmaker  is  assigned  to  each  one,  see  Figure 

3 .1 .  Each  DM  is  capable  of  observing  only  the  points  that  appear  in  his 
subsector.  He  can  assess  the  situation,  i.e.,  estimate  the  trajectory,  and 
select  the  response,  i.e.,  which  weapons  to  fire,  for  targets  with 
trajectories  totally  within  his  subsector.  This  is  the  case  when  two  points 
that  define  the  target  are  within  his  subsector.  Since  it  is  possible  for 
trajectories  to  •straddle"  the  subsector  boundaries,  it  is  necessary  that  SA 
information  be  shared.  Thus,  DM1  and  DM*  share  information  that  relates  to 
their  common  boundary.  Similarly,  DM1  and  DM*  share  Information  that  relates 
to  targets  that  cross  their  common  boundary.  To  keep  the  example 
computationally  simple,  the  situation  assessment  stages  of  DM1  and  DM*  are 
assumed  to  contain  a  single  algorithm  fn;  that  of  DM*  contains  two  algorithms 
f*  and  f*.  In  contrast,  the  response  selection  stage  of  DM*  contains  a  single 
algorithm  h*.  while  the  RS  stages  of  DM*  and  DM*  contain  two  algorithms  h“  and 
h^,  n  -  1,3.  Therefore,  the  internal  decision  strategies  are  p(u*),  p(v*|l*), 
and  p(v*|Z*).  The  detailed  structure  of  this  organization  is  shown  in  Figure 

3.2.  Note  that  the  interactions  defined  by  the  information  structure  exhibit 
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no  loops  or  cycles.  In  particular,  consider  the  SA  information  shared  between 
DM1  and  DM*.  The  variables  z1  and  z12  are  generated  as  a  result  of  DM2's 
processing  of  x1.  Similarly,  z2  and  z11  are  produced  by  DM2,  Once  DM2  has 
forwarded  x21  to  DM1,  the  final  assessment  z1  can  be  determined  using  the 
information  fusion  (IF)  stage  of  DM1.  DM2  determines  his  value  of  z2  after 
receiving  z12,  z*2  and  after  producing  z2 .  While  precise  timing  is  not 
explicitly  required,  it  is  implicit  in  that  a  particular  stage  of  processing 
cannot  be  completed  until  ail  the  requisite  input  variables  are  received. 
Furthermore,  it  is  assumed  that  the  overall  input-output  processing  of  the 
organization  can  be  accomplished  on  the  average  within  the  mean  interarrival 
time  t. 

The  second  organizational  structure.  Organization  B,  incorporates  a 
decisionmaker  who  has  a  supervisory  role.  It  is  defined  as  follows:  the 
rectangular  sector  is  divided  into  two  equal  subsectors  for  which  DM1  and  DM* 
are  responsible  for  assessing  the  threat;  however,  data  from  the  area  adjacent 
to  the  boundary  betweem  DM1  and  DM*  are  transmitted  to  the  coordinator  or 
supervisor,  DM2,  who  resolves  conflicts  and  assigns  targets  either  to  DM1  or 
to  DM*  as  appropriate  (Fig.  3.3).  This  is  accomplished  through  command  inputs 
v21  and  v22  from  the  coordinator  to  the  two  commanders.  They  in  turn  exercise 
their  response  y1  and  y*.  respectively.  Again,  for  computational  simplicity, 
it  is  assumed  that  DM1  and  DM1  have  a  single  algorithm  fn,  for  their  SA  stage 
and  two  algorithms  h“  and  h”  for  the  RS  stage.  The  coordinator,  DM2,  has  an 
algorithm  IF2  for  processing  the  assessed  situations  z12  and  z*2  and  two 
algorithms,  h2  and  h2 ,  in  the  RS  stage.  The  internal  decision  strategies  are 
p(v1|z1),  p(v2|z2),  and  p(v*|z>).  The  structure  of  this  organization  is  shown 
in  Figure  3.4.  Again,  note  that  the  information  structure  is  indeed 
acyclical . 


Figure  3.3.  Two  Sectors  (Hierarchical  Organization) 


In  order  to  avoid  complex  protocols,  it  is  assumed  that  no  more  than  one 
threat  can  be  in  each  sector  and  there  can  be  only  one  threat  that  crosses 
sectors  at  a  time.  These  assumptions  avoid  two  conflicts:  (a)  one  DM  having 
to  decide  about  two  threats  at  the  same  time  and  (b)  the  center  DM  receiving 
data  from  the  other  two  DMs  and  having  to  assess  two  threats  simultaneously. 
Actually,  the  second  assumption  can  be  relaxed  so  that  two  threats  that  cross 
boundaries  can  be  present  provided  they  do  so  on  different  boundaries  and  the 
net  result  is  such  that  a  single  DM  does  not  have  to  respond  to  both  of  them. 


Figure  3.4.  Organization  B:  Hierarchical  Structure 

Fourteen  different  threat  configurations  are  possible.  For  the 
evaluation  of  the  performance-workload  locus,  fifty  six  distinct  threat  sets 
were  used  where  sets  were  distinguished  by  whether  the  threat  was  fast  or  slow 
and  by  what  the  direction  of  the  threat  was.  The  objective  probability 
distribution  p(x)  was  taken  to  be  non-uniform.  In  order  to  provide  some 
contrast  and  also  represent  a  plausible  situation,  the  probability 
distribution  q(x)  perceived  by  the  three  decisionmakers  was  based  on  the 
assumption  that  the  DMs  do  not  take  into  account  that  threats  can  cross 
sectors;  they  assign  zero  probability  to  such  events.  On  the  other  hand,  they 
assume  a  uniform  probability  distribution  for  the  various  configurations  that 
do  not  include  threats  crossing  sectors . 
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The  level  of  activity,  measured  by  whether  a  decisionmaker  expects  to 
respond  to  a  threat  or  not  is  shown  in  Table  1.  The  active  status  denotes 
either  autonomous  operation  — the  DM  receives  a  threat  in  his  sector  and 
responds  to  it  —  ot  interaction  with  another  DM  by  sharing  situation 
assessment  information. 


TABLE  1 .  Expected  Activity  by  DMs  for  p(x)  and  q(x) 


DM 

STATUS 

p(x) 

ORG  A 

ORG  B 

q(x) 

ORG  A 

ORG  B 

1 

Active 

64% 

82% 

50% 

65% 

Inactive 

36% 

18% 

50% 

35% 

2 

Active 

79% 

50% 

30% 

30% 

Inactive 

21% 

50% 

70% 

70% 

3 

Active 

64% 

82% 

50% 

65% 

Inactive* 

i  36% 

18% 

50% 

35% 

3.6  RESULTS 

The  consistency  measure  Q  for  the  parallel  organization  A  and  for  task 
probability  distributions  p(x)  and  q(x)  is  shown  in  Figures  3.5  and  3.6,  while 
Figures  3.7  and  3.8  are  the  corresponding  plots  for  a  hierarchical 
Organization  B. 

Figure  3.5  shows  that  there  are  no  strategies  that  the  organization 
members  can  use  that  will  not  lead  to  overload,  if  the  interarrival  time  x  is 
less  than  29  units  of  time.  Furthermore,  if  t  is  more  than  43  units,  then  no 
strategy  will  overload  the  DMs.  The  fraction  of  strategies  that  satisfy  the 
bounded  rationality  constraints  is  a  non-decreasing  function  of  r,  as  shown  by 
the  gradual,  step  like  increases  in  Q.  Similarly,  there  are  no  strategies 
that  will  yield  a  cost  that  is  less  than  3.2  units,  while  if  the  constraint  is 
that  the  cost  be  no  more  than  3.4  units,  then  all  strategies  are  feasible^ 
This  is  indicated  in  Figure  3.5  by  the  steepness  of  the  slope  in  the  J 
direction.  Now  consider  Figure  3.6.  This  shows  the  effect  of  underestimating 
the  task  requirements  by  the  decisionmakers.  By  neglecting  to  take  into 
consideration  the  common  occurence  of  threats  that  cross  over  from  one  sector 
to  another  and  the  resulting  need  for  communication  and  coordination,  the 
decisionmakers  perceive  that  they  can  handle  threats  arriving  every  14  units 
of  time  on  the  average.  All  strategies  are  feasible,  if  the  threats  arrive 
every  28  or  more  units  of  time.  On  that  basis,  they  can  choose  strategies 
they  believe  to  be  feasible,  but  which  in  reality  will  cause  overload  and, 
consequently,  a  degradation  of  performance,  possibly  a  severe  one. 


Figure  3.5.  Consistency  Measure  Q  for  Organization  A:  p(x) 


Figure  3.6.  Consistency  Measure  Q  for  Organization  A:  q(x) 
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Under  this  assumption,  it  is  also  clear  that  performance  will  degrade. 
Indeed,  there  are  no  strategies  that  will  yield  a  cost  of  less  than  3.5  units. 
Superposition  of  Figure  3.6  on  Figure  3.5  shows  clearly  the  effect  of  q(x)  . 
The  DMs  assume  that  the  task  is  simpler  than  it  really  is  and  consequently, 
select  inappropriate  strategies  that  lead  to  high  workload  and  degraded 
performance . 

Very  similar  observations  can  be  made  by  comparing  the  Q  loci  of 
Organization  B,  Figures  3.7  and  3.8.  In  that  case,  too,  the  Q  locus  for  the 
objective  probability  is  farther  from  the  origin  than  the  locus  resulting  when 
the  perceived  uncertainty  by  the  decisionmakers  is  used.  The  decisionmakers 
have  underestimated  the  need  for  coordination  to  handle  threats  that  occur 
near  or  on  the  boundary  between  the  two  sectors. 

The  assumption  that  a  decisionmaker  may  select  any  of  the  admissible,  but 
no  necessarily  feasible,  strategies  with  equal(  probability  is  not  a  realistic 
one.  If  the  DM  thinks  that  a  certain  subset  £s  of  the  decision  strategy  space 
7  will  meet  the  performance  and  workload  constraints,  then  he  will  choose 
strategies  from  that  set  only  and  ignore  the  rest.  This  can  be  modeled  by 
assuming  a  uniform  probability  distribution  for  the  strategies  in  Zs  and 
assigning  zero  probability  to  the  strategies  not  in  7  .  If  a  decisioipaker 's 
perception  of  the  task's  probability  distribution  q(x)  is  such  that  £s  is  the 
null  set,  then  a  uniform  probability  distribution  is  assumed  for  selecting  a 
strategy  from  the  set  £. 

If  the  decisionmaker's  q(x)  is  the  same  as  the  designer's,  the  objective 
probability  distribution  p(x),  then  the  feasible  strategy  set  is  denoted  by 
7g .  The  relative  position,  of  the  two  strategy  sets,  the  one  due  to  q(x), 
and  the  one  due  to  p(x)  ,  £0 ,  may  be  such  that  the  two  sets  are  disjoint, 
overlapping,  or  identically  the  same.  Therefore,  in  selecting  a  strategy  from 
Ts,  the  decisionmaker  may  be  selecting  a  strategy  that  meets  the 
specifications  of  the  task  as  seenrby  the(designer  or  he  may  not.  In  order  to 
analyze  the  relationship  between  Zs  and  £0,  the  consistency  measure  introduced 
in  Eq.  (3.12)  can  be  modified  as  follows: 

(  V(^)/V(T)  for  V(Zg)  -  0 

)  V(l')  V(l') 

Q(J.t)  =  < - 5 -  for  V(7  )  =  0  (3.15) 

v(Zs) 

1  for  V(Io')  -  V(Z) 


where  V(*)  is  again  the  volume  in  the  strategy  space.  The  quantity  Q  can  be 
interpreted  as  the  probability  that  a  decisionmaker,  who  perceives  the 
probability  distribution  of  the  tasks  are_being  q(x),  will  select  a  decision 
strategy  that  satisfies  the  requirements  (J,t). 


3 .7  CONCLUSION 


In  this  chapter,  an  approach  to  the  evaluation  of  alternative 
organizational  designs  has  been  presented.  The  issues  that  arise  when  the 
designer's  and  the  decisionmakers '  knowledge  of  the  tasks  and  of  their 
relative  value  differ  have  been  discussed.  A  measure  of  organizational 
effectiveness  has  been  introduced,  the  consistency  measure  Q,  for  the  case 
when  the  designer  and  the  decisionmakers  differ  in  their  perception  of  the 
uncertainty  associated  with  the  inputs  or  tasks.  Two  three-person 
organziations ,  one  with  parallel  structure  and  one  with  a  hierarchical  one, 
have  been  used  to  illustrate  the  approach. 

One  weakness  of  the  procedure  is  that  the  analytical  formulation  of  the 
workload  does  not  yield  much  insight  to  the  differences  between  the  two  points 
of  view.  Alternatively,  the  computational  approach  yields  results  that  are 
specific  to  each  case  and  problem  that  are  being  studied.  Thus,  upon 
completion  of  this  study,  a  major  effort  was  initiated  to  develop  efficient 
computational  algorithms  and  implement  them  on  a  workstation  so  that  many 
cases  can  be  studied  with  ease.  Such  a  workstation  has  now  been  developed  and 
will  be  used  for  further  studies  and  analyses. 
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4.  MEASURES  OF  EFFECTIVENESS  IN  COMMAND  AND  CONTROL* 

In  the  previous  chapter,  a  methodology  was  presented  for  analyzing  and 
comparing  alternative  organizational  forms.  In  the  language  of  the  theories 
for  system  evaluation,  two  measures  of  performance  (MOPs),  accuracy  and 
workload,  were  introduced  and  the  MOP  locus,  the  performance-workload  locus, 
was  defined.  Then,  a  measure  of  effectiveness  (MOE) ,  the  measure  of 
consistency  Q,  was  defined  and  evaluated  on  the  basis  of  the  MOP  locus.  These 
concepts  are  quite  general  and  translate  easily  to  the  evaluation  or 
assessment  of  C*  systems.  One  of  the  tasks  in  the  research  effort  was  to 
approach  the  problem  of  evaluation  from  both  sides  -  from  the  organization 
theory  perspective  and  from  the  system  design  perspective.  The  long  term 
objective  has  been  to  develop  a  unified  approach. 

In  this  chapter,  a  methodology  for  measuring  the  effectiveness  of  C* 
systems  is  described  and  is  then  applied  to  an  abstraction  of  a  C*  system  of 
the  US  Army. 

4.1.  INTRODUCTION 

Improvements  in  weapon  system  technology,  and  higher  capacity  and  speed 
in  data  transmission,  combined  with  an  increasing  complexity  of  the 
battlefield,  impose  severe  time  constraints  on  hardware,  software,  and  human 
decisionmakers.  The  purpose  of  this  task  was  to  extend  the  methodology 
presented  in  Boutbonnier  and  Levis  [23]  to  consider  measures  of  performance 
(MOPs)  and  measures  of  effectiveness  (MOEs)  that  include  time. 

Time  has  always  been  of  crucial  importance  in  combat  and,  consequently, 
to  command  and  control  [24] .  As  Lawson  [25]  relates,  "in  a  typical  discussion 
of  Command  and  Control,  it  is  taken  as  axiomatic  that  the  information 
presented  to  the  commander  must  be  'timely'  as  well  as  accurate,  complete, 
etc.  Little  or  nothing  is  said  about  how  timely  is  timely  enough;  nor  is  any 
yardstick  given  by  which  to  measure  'timeliness'.  Rather,  the  clear 
implication  is  that  all  would  be  well  if  only  communications  and  computers 
were  'faster'.  In  addition,  this  attention  to  rates  (e.g.,  information 
processing  rates,  rate  of  fire,  etc.)  in  which  time  only  appears  in  the 
denominator,  ha3  led  to  a  preoccupation  with  the  performance  characteristics 
of  the  component  parts  of  a  C*  system.  It  does  not  provide  any  means  of 
comparing  the  effect  of  an  increase  in  one  'rate'  with  that  of  an  increase  in 
some  othe  rate." 

The  methodological  framework  is  the  one  developed  by  Boutbonnier  and 
Levis  [23].  The  aspects  that  time  can  take  in  a  warfare  environment  are 
numerous.  The  ones  that  this  assessment  methodology  consideres  are: 
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System  response  time.  It  characterizes  the  time  delay  between  the  moment 
when  the  C"*  system  receives  a  stimulus  and  the  moment  it  can  deliver  a 
response.  It  is  the  sum  of  all  the  time  delays  at  every  level  of  the  process. 

Tempo  of  operations .  In  most  military  situations,  rates  are  used  to 
express  the  important  quantities,  e.g.,  rounds  per  minute,  miles  per  hour. 
The  term  in  commom  usage  for  the  operating  rate  of  a  C*  system  is  its  "tempo". 
Lawson  [25]  defines  it  as  the  number  of  actions  per  unit  of  time  which  the 
system  is  executing  and  states,  further,  that  "the  tempo  tells  us  how  complex 
an  environment  the  system  can  handle  (i.e.,  its  bandwidth)  while  the  response 
time  tells  us  when  it  responds  in  time  (i.e.,  the  phase  delay  in  the  system)". 

These  notions  depend  on  what  is  actually  taking  place,  i.e.,  the 
scenario.  The  event  that  stimulates  the  C*  system  is  only  the  partial 
perception  by  the  system  of  a  global  scenario.  Different  scenarios  can  be 
perceived  through  identical  events,  and  the  system  is  confronted  with 
uncertainty.  Once  the  scenario  is  identified  with  enough  certainty,  then  an 
option  must  be  selected.  It  appears  that  any  assessment  of  a  C*  system  must 
consider  the  crucial  role  of  the  scenario  and  the  tempo  of  operations  it 
implies:  to  each  scenario  corresponds  an  evaluation  of  the  effectiveness  of 
the  system.  These  partial  evaluations  can  be  merged  into  an  overall  measure 
of  effectiveness  for  a  given  range  of  possible  scenarios. 

Timeliness  is  defined  here  as  a  system's  ability  to  respond  within  an 
allotted  time.  Allotted  time,  in  this  context,  is  the  time  interval  over 
which  our  forces,  which  include  a  C*  system  being  assessed,  can  affect  the 
environment.  This  allotted  time  is  described  by  a  window  of  opportunity  whose 
parameters  are  determined  by  the  system  and  the  mission.  Two  quantities  are 
needed  to  specify  the  window  of  opportunity:  one  choice  is  the  lower  and  the 
upper  bound  of  the  time  interval,  t  and  t  ,  respectively;  another  one  is  one 
of  the  bounds  and  the  length  of  the  interval,  e.g.,  t  and  At.  Such  a  time 
interval  is  not  sufficient  to  yield  by  itself  a  measure  of  effectiveness;  one 
must  also  consider  the  way  this  time  is  employed,  i.e.,  one  must  consider  the 
doctrine  that  is  in  effect,  was  well  as  the  tactics  that  are  feasible  in  such 
an  interval. 

The  methodology  used  in  this  paper  is  based  on  six  concepts:  system, 
environment,  context,  parameters,  measures  of  performance,  and  measures  of 
effectiveness.  The  first  three  are  used  to  pose  the  problem,  while  the  last 
three  define  the  key  quantities  in  the  analytical  formulation  of  the  problem. 
The  analytical  aspects  of  the  methodology  address  mainly  the  relationships 
between  hardware  characteristics,  system  structure,  and  standard  operating 
procedures  (SOPs)  to  system  performance. 

The  system  consists  of  components,  their  interconnections,  and  a  set  of 
operating  procedures .  A  boundary  can  be  drawn  that  defines  what  is  included 
within  the  system  whose  effectiveness  is  to  be  assessed.  What  is  included 
depends  on  the  analysis  at  hand.  The  environment  consists  of  our  own  forces 
and  the  adversary's  forces  upon  which  our  forces  can  act  and  which  can  act 
upon  ours.  For  example,  the  C*  system  is  used  to  direct  forces  and  monitor 


(sense)  the  environment.  An  engagement  between  two  forces  in  an  urban  area  or 
at  a  mountain  pass  define  typical  environments.  A  C*  suDsystem  of  a  fire 
support  system  or  a  communication  network  are  typical  systems.  The  context 
denotes  the  set  of  conditions  and  assumptions  within  which  the  system  and  the 
environment  exist.  The  relationship  between  system,  environment,  and  context 
are  shown  in  Figure  4.1. 
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Figure  4.1.  System,  Environment  and  Context 

Parameters  are  the  independent  quantities  used  to  specify  the  system  and 
the  mission  requirements.  For  example,  in  the  case  of  a  fire  support  system, 
system  parameters  may  include  quantities  that  describe  the  detection 
equipment,  computational  time  delays,  kill  radius  of  the  munition,  and  failure 
probabilities  associated  with  the  components,  to  name  but  a  few.  Parameters 
of  the  mission  may  be  the  tempo  of  operations ,  as  described  by  the  speed  of 
the  threats,  and  the  size  of  the  engagement. 

Measures  of  Performance  are  quantities  that  describe  system  properties  or 
mission  requirements.  MOPs  for  a  command  and  control  system  may  include 
reliability,  survivability,  cost,  and  probability  to  kill.  The  mission 
requirements  should  be  expressed  by  the  same  quantities  as  the  system  MOPs, 
e.g.,  minimum  reliability  or  survivability,  maximum  cost,  or  minimum 
probability  to  kill .  System  parameters  are  defined  within  the  system 
boundary;  MOPs  may  be  defined  within  the  boundary  or  they  may  include  aspects 
of  the  environment. 

Measures  of  Effectiveness  (MOEs)  are  quantities  that  result  from  the 
comparison  of  the  system  MOPs  to  the  mission  requirements.  They  reflect  the 
extent  to  which  the  system  meets  the  requirements .  To  evaluate  the  MOEs,  it 
is  necessary  to  go  outside  the  boundary  and  consider  the  environment.  Only 


then  could  the  effect  of  the  system  on  the  mission  outcome  be  evaluated . 
These  definitions  of  parameters,  MOPs,  and  MOEs  are  consistent  with  those 
developed  at  a  recent  MORS  Workshop  on  the  subject  [26] . 


In  this  methodology  for  assessing  effectiveness,  the  system  HOPs  and 
mission  requirements  must  be  modeled  and  analyzed  independently,  but  in  a 
common  context.  The  system  capabilities  should  be  determined  independently  of 
the  mission  requirements,  and  the  mission  requirements  should  be  derived 
without  considering  the  system  to  be  assessed.  Otherwise,  the  assessment  is 
biased.  Since  the  steps  of  the  methodology  have  been  presented  in  [23],  they 
and  the  modeling  of  timeliness  will  be  described  here  through  application  to 
an  idealized  tactical  fire  direction  system.  This  problem  is  formulated  in 
Section  4.2;  in  Section  4.3,  the  system  and  the  mission  are  modeled  and  their 
respective  loci  defined.  In  Section  4.4,  a  class  of  measures  of  effectiveness 
is  defined,  while  in  Section  4.5  the  results  are  presented  and  discussed. 

4.2  SYSTEM  AND  MISSION 

The  concepts  discussed  in  Section  4.1  can  be  illustrated  by  applying  the 
methodology  for  System  Effectiveness  Analysis  to  an  'electronically  integrated 
command  and  control  information  system  that  also  processes  fire  missions" 
[27]  . 

One  can  isolate  three  main  elements  in  a  fire  support  system  at  the 
battalion  level:  the  forward  observer,  the  battalion  fire  direction  center, 
and  the  field  artillery  cannon  battery.  The  system  can  include  several 
forward  observers  and  several  batteries  connected  to  the  same  central 
battalion  computer. 

The  Forward  Observer  (FO)  receives  the  initial  stimulus  by  detecting  an 
enemy  threat.  The  FO  is  equipped  with  vehicle  position  determining  equipment 
and  a  laser  rangefinder.  These  allow  the  FO  to  locate  accurately  area  targets 
for  fire  and  to  conduct  one-round  adjustments .  The  FO  is  also  equipped  with 
the  Digital  Message  Device  (DMD),  a  portable,  battery-powered  device  that 
transmits  and  receives  digital  bursts.  The  FO  uses  the  DMD  to  communicate 
estimates  of  the  position  and  velocity  of  the  target,  and  requests  for  fire  to 
the  battalion  computer. 

The  battalion  Fire  Direction  Center  (BN  FDC)  is  provided  with  a  central 
computer.  Digital  communication  over  any  standard  communication  means  (radio 
or  wire)  provides  for  input  of  data  into  the  computer  center  and  for  the 
return  of  the  results.  Forward  observers  and  firing  batteries  are  provided 
with  remote  terminal  equipment  to  obtain  data  from  the  central  computer. 

The  Battery  Display  Unit  (BDU)  is  the  cannon  battery's  link  with  the  C* 
system.  Each  battery  has  one  BDU.  The  BDU  assists  execution  of  fire  plans  by 
receiving  and  printing  firing  data  for  each  target  that  the  battery  will  fire. 
This  receive-only  unit  initiates  an  automatic  acknowledgement  that  is  sent 
back  to  the  FDC  computer. 

While  this  is  the  basic  configuration,  additional  equipment  is  maintained 
in  parallel  to  augment  the  basic  system.  Voice  communication  links  can  be 
added  in  parallel  with  the  digital  links,  for  instance,  between  the  battalion 
fire  direction  center  and  the  cannon  battery.  Voice  communication  is  slower, 
more  vulnerable,  but  still  very  useful,  if  the  digital  link  fails.  If  the 


fire  support  system  computer  fails  at  the  battalion  level,  the  battery  has  the 
capacity  to  do  the  firing  computations  locally.  This  alternative  is  slower, 
though. 

A  schematic  representation  of  the  system  that  will  be  analyzed  is  shown 
in  Figure  4.2.  Seven  stages  are  shown.  In  this  model,  nodes  are  not  subject 
to  failure;  only  links  are.  Alternative  implementations  of  each  stage  are 
shown  by  parallel  links;  e.g.,  a  voice  link  is  in  parallel  with  the  digital 
link  between  the  battalion  fire  direction  center  and  battery  B. 
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Figure  4.2.  Fire  Support  System  Structure 

If  the  BN  FDC  computer  does  not  work,  the  target  estimates  from  the  FO 
can  be  sent  to  battery  B  through  voice  communication  (the  BN  FDC  acts  as  a 
simple  relay) .  The  battery  crew  can  then  compute  the  firing  data  manually 
with  the  Field  Artillery  Digital  Automatic  Computer  (FADAC).  In  the  case 
where  the  firing  data  are  computed  at  the  BN  FDC  level  and  transmitted  by 
voice  communication  to  battery  B,  neither  the  BDU  nor  the  manual  technique 
have  to  be  used.  The  voice  communication  of  the  firing  data  reaches  directly 
the  firing  platform  of  the  battery.  In  order  to  assess  properly  the 
effectiveness  of  this  system,  it  is  necessary  to  specify  the  environment  and 
context  in  which  it  operates,  i.e.,  define  the  scenario. 

The  idealized  scenario  that  will  be  considered  is  shown  in  Figure  4.3. 
Some  vital  node  of  our  forces'  C1  system  is  situated  at  the  end  of  a  valley. 
A  road  along  this  valley  leads  to  this  node.  The  topography  of  the  area  is 
perfectly  known  by  our  forces,  and  the  road  is  the  only  access  to  the  node.  A 
fire  support  battalion  including  one  forward  observer  (FO),  one  battalion  fire 
direction  center  (BN  FDC),  and  two  batteries  Bx  and  Ba  has  been  positioned  to 
protect  this  access.  This  battalion  is  equipped  with  the  fire  support  system 
defined  in  Figure  4.2.  The  batteries  cannot  see  the  road:  they  shoot 
according  to  the  firing  directions  that  are  computed  on  the  basis  of  the 
observer's  estimates. 

An  enemy  tank  (threat)  appears  in  the  field  of  vision  of  the  forward 
observer.  It  is  moving  on  the  road  towards  our  forces  with  hostile 
intentions.  The  mission  is  to  defend  the  node,  i.e.,  to  prevent  the  tank  from 
attacking  the  node. 
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Figure  4.3.  Scenario 


It  is  assumed  that  the  threat  cannot  attack  the  fire  support  battalion 
directly;  the  only  countermeasure  that  will  be  considered  is  the  Jamming  of 
communications  by  the  enemy.  It  is  also  assumed  that  the  threat  will  pursue 
its  attack,  even  after  it  is  fired  upon.  It  will  try  to  carry  out  its  own 
offensive  mission,  as  if  it  encountered  no  reaction  from  our  forces.  This  is 
a  highly  idealized  situation.  It  is  used  to  illustrate  the  methodology, 
rather  than  represent  current  practice  in  terms  of  doctrine  or  tactics. 

The  problem  of  interest  is  to  determine  how  effective  this  particular 
command  and  control  system  is.  But,  as  indicated  earlier,  to  assess 
effectiveness,  one  has  to  go  outside  the  boundaries  of  the  C1  system  itself 
and  consider  the  mission  and  the  environment.  In  this  case,  it  is  postulated 
that  the  system  to  be  assessed  consists  of  the  forward  observer,  the  BN  FDC, 
the  BDU,  the  FADAC,  and  the  associated  communication  links.  The  batteries 
themselves,  as  well  as  the  threat  and  the  node  to  be  defended,  constitute  the 
environment;  the  terrain  constitutes  the  context.  Note  that  in  applying  this 


methodology,  a  choice  has  to  be  made  each  time  as  to  what  is  included  within 
the  boundaries  of  the  system.  In  this  case,  the  sensors  are  included,  as  are 
some  parts  of  the  weapons  system.  In  another  application,  the  whole  fire 
support  system  could  be  considered  as  a  integrated  weapons  system. 

The  system's  performance  will  be  characterized  by  three  MOPs^  Two  of 
them  define  the  window  of  opportunity,  namely,  the  upper  bound  t  and  the 
width  of  the  window  At.  The  third  one  is  the  overall  probability  of  kill 
(OPK)  that  measures  the  ability  of  the  total  fire  support  system  to  stop  the 
threat . 

For  each  value  of  the  selected  system  parameters,  a  value  for  the  triplet 
(t  ,  At,  OPK)  is  obtained.  As  the  parameters  are  allowed  to  vary  over  their 
respective  range  of  values,  a  locus  is  constructed  in  the  space  of  MOPs .  This 
is  defined  as  the  system  locus,  Ls . 

The  mission  requirements  in  this  case  are  expressed  simply  as  the  minimum 
acceptable  probability  that  the  node  will  be  defended  successfully,  i.e.,  that 
the  threat  will  be  prevented  from  firing  at  the  node.  This  requirement  can 
also  be  expressed  as  a  locus  in  the  MOP  space;  it  is  the  mission  locus  Lr. 
Effectiveness  for  this  system  and  mission  is  defined  by  how  well  the  overall 
kill  probability  meets  the  mission  requirement. 

In  the  next  section,  the  mathematical  models  of  the  system  and  the 
mission  will  be  developed  and  the  corresponding  loci  constructed. 

4.3  SYSTEM  AND  MISSION  MODELS 

Each  node  and  each  link  of  the  C*  system  is  assumed  to  have  a  probability 
of  failure,  independently  of  the  countermeasures  taken  by  the  enemy.  These 
component  failure  probabilities  determine  the  system's  reliability.  Since  the 
system  is  operating  in  a  hostile  environment,  the  communication  links  are 
subject  to  jamming  by  the  enemy.  The  probability  of  failure  due  to  Jamming 
determine  the  system's  survivability.  Although  the  two  concepts  of 
reliability  and  survivability  are  distinct  because  the  underlying 
probabilities  of  failure  have  different  causes,  in  this  analysis  they  will  be 
merged  to  reduce  the  dimensionality  of  the  problem.  A  single  vector  of 
probabilities,  g,  will  be  used  as  a  parameter  that  describes  the  failure 
characteristics  of  the  system's  components.  In  a  full  scale  analysis,  the  two 
will  be  separate. 

The  parameter  that  determines  the  tempo  of  operations  is,  in  this  case, 
the  speed  w  of  the  threat.  A  whole  range  of  different  versions  of  the 
scenario  can  be  investigated  by  varying  the  speed  w. 

It  is  assumed  that  the  main  source  of  uncertainty  is  the  estimate  of  the 
threat's  state  by  the  forward  observer.  If  the  FO  uses  two  sightings  to 
determine  the  threat's  state,  then  an  appropriate  system  parameter  can  be,  for 
example,  the  angle  0  that  separates  the  two  sightings.  Intuitively,  the 
larger  the  angle  0  the  more  accurate  the  speed  estimate,  but  the  longer  the 
response  time . 


Since  the  commander  has  a  choice  as  to  the  resources  he  may  use,  two 
cases  will  be  considered:  (a)  one  battery  (Bj),  and  (b)  two  batteries  (Bx  and 
B,).  In  the  second  case,  two  different  tactics  can  be  analyzed:  coordinated 
fire  and  uncoordinated  fire. 

Response  Time  and  Window  of  Opportunity 

The  geometric  relations  for  this  scenario  are  shown  in  Figure  4.4. 
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Figure  4.4.  Geometric  Relations  between  System  and  Threat 

To  begin  the  process  of  quantifying  the  timeliness  characteristics  of  the 
system  through  the  determination  of  the  window  of  opportunity,  consider  Figure 
4.5,  which  shows  the  chronological  sequence  of  the  response  process.  The  time 
that  a  round  fired  by  a  battery  hits  the  ground,  timpact,  is  given  by: 
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(4.1) 


Atjl  is  computed  from  the  geometry  in  Figure  4.4.  It  is  a  function  of  the 
speed  w,  the  angle  0,  and  the  observation  time  [28]. 
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Figure  4.5.  Time  Profile  of  the  System  Response 

A  sensitivity  analysis  has  shown  that  At,  can  be  taken  as  constant  for 
this  geometry  and  characteristics  of  the  weapon  system.  For  the  values  chosen 
for  this  example,  this  constant  is  36  seconds. 

Let  the  lower  bound  of  the  window  of  opportunity,  t*,  be  equal  to  the 
minimum  impact  time.  For  a  given  angle  p  and  a  given  target  velocity  w,  the 
earliest  impact  time  is  determined  by  the  earliest  possible  observation  time , 
i.e.,  tQbs  =  0,  and  by  the  minimal  time  delay  (At , )  mi  n  between  the  end  of  the 
estimation  and  the  actual  firing  of  the  battery.  Thus: 

t*  =  At  (w.p ,0)  +  (At  )  +  At  (4.3) 
i  *  min  * 


This  earliest  impact  time  may  be  considered  as  the  response  time  of  the 
system.  Note  that  it  depends  on  the  scenario,  the  sensor  characteristics,  and 
the  positioning  of  the  forward  observer  and  the  batteries  with  respect  to  the 
threat . 

Let  be  the  point  on  the  trajectory  where  the  threat  leaves  the  area 
covered  by  battery  Bx  (see  Fig.  4.4)  in  the  single  battery  case,  or  the  area 
covered  by  either  batter^in  the  two  battery  case.  Let  the  upper  bound  of  the 
window  of  opportunity,  t  ,  be  the  time  that  the  threat  moves  past  point  . 
This  bounds  from  above  the  admissible  impact  times,  i.e.,  only  impact  times 
occuring  within  the  window  of  opportunity  are  allowed: 


The  resulting  time  interval.  At,  is  the  system  window  of  opportunity:  the 
system  can  deliver  a  response  to  the  stimulus  at  any  time  tim pacv  between  t 
and  t**  (for  t*  <  t**).  The  window  of  opportunity's  completely  characterized 
by  the  ordered  pair  (t  *,  At),  where  At  =  t  -  t  .  Changes  in  the  values  of 
the  parameters  w,  and  0  lead  to  changes  in  the  window  of  opportunity. 

Performance 


The  single  shot  kill  probability  SSPK  associated  with  an  impact  time  is 
easily  computed  by  taking  into  account  the  uncertainty  in  the  speed  estimate, 
and  the  kill  radius  of  the  munition.  For  fixed  values  of  w  and  tobg,  the 
shape  of  the  variations  of  SSPK  with  t  is  given  in  Figure  4.6;  the  latter  also 
shows  an  important  trade-off.  As  0  increases,  the  width  of  the  window  of 
opportunity  decreases  because  it  takes  a  longer  time  for  the  FO  to  obtain  his 
estimate.  But  at  the  same  time,  a  large  0  yields  a  more  accurate  estimate  of 
the  speed  of  the  target.  Therefore,  the  kill  probability  is  increased.  The 
upper  limit  t  is  unaffected  by  changes  in  0 . 


Figure  4.6.  Single  Shot  Kill  Probability  as  a  Function  of  Impact  Time 

Also,  as  time  goes  by,  if  there  are  no  updates  of  the  threat's  state,  the 
uncertainty  of  its  exact  position  increases  and,  therefore,  SSPK  decreases 
with  time . 

The  seven-stage  structure  of  the  C*  system  has  been  presented  in  Figure 
4.2.  The  corresponding  tree  that  represents  the  system  states.  Figure  4.7, 
reveals  that  out  of  the  ten  possible  paths,  six  paths  do  not  lead  to  the 
transmission  of  information  from  the  forward  observer  to  the  batteries . 
For  each  of  the  four  paths  that  indicate  successful  communication,  the 
following  quantities  are  defined: 


q(i):  probability  that  the  path  #i  is  operational;  i  =  1,2, 3. 4. 


y ( i )  =  (AtjJj^d)  +  At,,  i.e.,  y(i)  is  the  minimum  time  delay  between 

the  estimates  by  the  FO  and  the  impact  time. 

6(i):  minimum  time  delay  necessary  to  recompute  new  firing  data  based 

on  the  initial  estimates,  to  transmit  them  and  to  set  up  the 
battery  accordingly.  If  the  system  recomputes  the  firing  data 
immediately  after  each  shot  and  fires  in  sequence,  then  6(i) 
represents  the  minimum  time  delay  between  two  shots. 
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Figure  4.7.  Tree  Representing  System's  Operational  States 

The  earliest  response  time  to  the  stimulus  is  t*.  The  system  can  use  the 
remaining  time  within  the  window  of  opportunity  to  deliver  other  responses, 
e.g.,  to  fire  again,  therefore  increasing  the  overall  kill  probability.  This 
can  be  done  in  many  different  ways.  This  analysis  focuses  on  two  of  them, 
which  are  classical  military  doctrines,  known  as  'LOOK-SHOOT-SHOOT- 
SHOOT  . . . ' [LSS]  and  'LOOK-SHOOT-LOOK-SHOOT . . . ■ [LSLS] . 

Doctrine  1;  'LOOK-SHOOT-SHOOT-SHOOT...'  The  observer  initially  makes 
estimates  of  the  speed  and  position  of  the  threat,  and  then  the  battery  keeps 
on  shooting  at  the  target,  recomputing  each  new  firing  data  on  the  basis  of 
these  initial  estimates. 
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The  observation  time  t0j)S  is  the  same  for  each  shot,  since  there  is  no 
updating  of  the  estimates.  The  time  delay  between  two  shots  is  thus  the 
interval  6.  The  battery  fires  as  many  shots  as  possible  within  the  window  of 
opportunity,  since  there  is  no  feedback  from  the  observer. 

Doctrine  2:  • LOOK -SHOOT-LOOK-SHOOT ... "  After  each  shot,  if  the  threat  is 
neither  destroyed  nor  incapacitated,  the  observer  makes  new  estimates  of  its 
speed  and  position,  new  firing  data  are  computed  on  the  basis  of  these  updated 
estimates,  the  battery  (or  batteries)  shoots  according  to  these  new  firing 
data,  and  so  on  until  the  upper  limit  of  the  window  of  opportunity  is  reached. 

The  overall  probability  of  kill  (OPK)  can  be  computed  from  the  single 
shot  probability  of  kill,  as  follows: 


SSPK (t 


impact#n 


(4.6) 


where  n  is  the  number  of  shots  possible  for  a  given  doctrine  and  a  given 
window  of  opportunity.  The  single  shot  probability  is  given  by: 


SSPK (t .  )  =  - - - 

impact#n  tifflpact#n  -  tobs#n 


(4.7) 


where  the  function  £  depends  on  the  kill  radius  of  the  munition,  the  speed  of 
the  target,  and  the  accuracy  of  the  observation.  Similar  expressions  have 
been  derived  for  Doctrine  2  as  well  as  for  the  two  battery  case  with 
coordinated  and  uncoordinated  fire.  For  details,  see  Cothier  [281. 

In  calculating  OPK.  it  is  assumed  that  the  C*  system  operates  in  the  same 
mode  (i.e.,  the  same  path  is  used)  throughout  that  engagement.  Therefore, 
four  different  values  of  OPK  can  be  computed,  one  for  each  possible  path. 

4.4  SYSTEM  AND  MISSION  LOCI 

The  three  system  MOPs  (t**,  At,  OPK)  can  be  derived  for  each  path  i. 
However,  it  is  preferable  to  obtain  an  overall  probabilistic  description  of 
these  MOPs.  For  any  of  the  6  paths  (Fig.  4.7)  that  fail  to  transmit  the 
information  from  the  FO  to  the  batteries,  At  is  zero,  and  so  is  the  OPK. 

For  paths  #1  to  #4,  At^  and  OPK^  vary  according  to  what  doctrine  is 
chosen.  The  MOPs  to  consider  are  thus  the  expected  values  of  these 
quantities . 
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OPK  =  ECOPK  } 


(4.9) 


where  q(i)  is  the  probability  that  the  i-th  path  is  used.  The  values  of  q(i) 
is  obtained  by  using  structure  functions  and  the  failure  probabilities 
(Cothier  [28]).  From  now  on,  only  the  expected  values  of  At  and  OPK  will  be 
considered . 

The  dependence  of  the  system  MOPs  on  the  system  parameters  is  shown  in 
Figure  4.8.  Note  that  OPK  depends  directly  on  the  parameters  w,  0,  and  p,  on 
the  two  other  MOPs,  t  *  and  At  (i.e.,  the  window  of  opportunity),  and  on  the 
doctrine  used.  In  other  words,  the  parameters  are  mapped  twice  in  the  third 
MOP,  directly  and  indirectly. 
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Figure  4.8.  Mapping  of  the  System  Parameters  into  the  System  MOPs 

To  each  value  of  the  parameter  set  (w,p,p)  corresponds  a  point  in  the  MOP 
space  (t  ,  At,  OPK) .  Now  consider  all  the  allowable  values  that  the 
parameters  make  take: 


wmin  -  w  - 
^min  -  P  -  Pmax 
pmin  -  P  -  pmax 


(4.10) 


If  the  parameters  are  allowed  to  vary  over  their  admissible  ranges,  then  the 
variations  define  a  locus  in  the  MOP  space.  This  is  the  system  locus  L_. 
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Mission  Locus 


The  analysis  of  the  mission  in  this  case  is  quite  simple:  the  mission 
requirements  can  be  expressed  directly  at  the  MOP  level.  The  mission 
objective  reduces  to  a  single  requirement,  a  condition  on  the  third  attribute 
OPK.  If  X  is  the  minimum  probability  that  the  threat  will  not  be  able  to 
attack  the  node  being  defended,  then  the  mission  locus,  Lp,  is  the  region  in 
the  MOP  space  (t  ,  At,  OPK)  that  satisfies  the  inequality: 

X  <  OPK  <  1  (4.11) 


In  general,  this  would  not  be  the  case.  Substantial  analysis  would  be 
required  to  convert  mission  objectives  into  requirements  and  then  express  the 
latter  as  a  locus  in  the  MOP  space. 

In  this  section  the  system  locus  and  the  mission  locus  have  been  derived. 
The  timeliness  of  the  system  has  been  described  in  terms  of  the  window  of 
opportunity,  which,  together  with  the  performance  measure  OPK,  have  led  to  the 
system  locus.  In  the  next  section,  the  two  loci  will  be  used  to  define  and 
evaluate  MOEs . 

4.5.  MEASURES  OF  EFFECTIVENESS 

The  next  step  consists  of  comparing  quantitatively  the  system  MOPs  to  the 
mission  requirements,  using  the  geometric  relationship  between  the  two  loci, 
L„  and  L_,  in  the  MOP  space.  If  the  two  loci  do  not  overlap,  i.e.,  if 

Ls  °  Lr  =  *  (4.12) 


then  the  system  MOPs  do  not  satisfy  the  mission  requirements  for  any  operating 
state  of  the  system.  Consequently,  effectiveness  should  be  zero,  regardless 
of  which  measure  is  used.  On  the  other  hand,  if  the  two  loci  coincide,  i.e., 
if 


L  H  L  =  L  =  L 
s  r  s  r 


(4.13) 


then  the  system  and  the  mission  are  perfectly  matched,  and  effectiveness 
should  be  equal  to  one.  Three  cases  can  be  distinguished  when  the  two  loci 
have  points  in  common. 

(a)  Neither  locus  is  included  in  the  other. 


L  n  L,  ^  «  and  L  H  L  <  L  and  L  O  L  <  L  (4.14) 

3  r  s  r  s  srr 

In  this  case,  only  some  of  the  values  that  the  system  MOPs  may  take 
satisfy  the  mission  requirements.  Many  different  measures  can  be  used  to 
described  the  extent  to  which  the  system  meets  the  requirements.  Each  of 
these  measures  can  be  considered  as  an  MOE  which,  if  normalized,  takes  values 
in  the  open  interval  (0,1).  Let  V  be  such  a  measure.  Then  one  MOE  is  defined 
by 


E  =  V(L  O  L  )/ V(L  )  (4.15) 

1  s  r  s 

while  another  is 

E  =  V(L  n  L  )/ V(L  )  (4.16) 

2  srr 


(b)  The  system  locus  is  included  in  the  mission  locus: 


L  pi  L  =  L  (4.17) 

s  r  s 

The  measure  Ex  is  equal  to  unity  —  all  operating  conditions  of  the 
system  meet  the  requirements.  But  E,  is  less  than  one;  the  system  can  attain 
only  certain  values  of  the  requirements. 

(c)  The  mission  locus  is  Included  in  the  system  locus: 

L  n  L  =  L  (4.18) 

srr 


that  is,  the  system  meets  all  the  requirements,  but  also  has  operating 
conditions  that  do  not  meet  the  requirements.  Then  Ex  is  less  than  one,  and 
E,  is  equal  to  unity. 

These  measures  are  only  two  of  the  many  that  can  be  defined;  they  earn  be 
thought  as  partial  MOEs,  since  they  give  only  partial  information  about  the 
relationship  between  the  two  loci .  These  partial  measures  can  be  combined 
into  a  single  measure  through  the  use  of  a  utility  function: 

E  -  u(Ei.Ea,...,Ek)  (4.19) 


The  subjective  judgements  of  the  system  developers  and  the  users  can  be 
incorporated  into  the  effectiveness  analysis  through  the  selection  of  the 
partial  measures  and  the  utility  function. 
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4.6  EFFECTIVENESS  ANALYSIS  AND  COMPARISON  OF  DOCTRINES 


For  this  example,  only  the  first  partial  MOE  of  Eq.  (4.15),  Ea,  will  be 
used.  The  mission  locus  is  such  that  E2  is  very  small  and  does  not 
discriminate  between  different  cases. 

The  One-Battery  Case:  Comparison  of  Two  Doctrines 

Figures  4.9  and  4.10  show  the  system  locus  and  its  intersection  (shaded 
region)  with  the  mission  locus  for  each  doctrine.  The  ratio  of  the  shaded 
volume  over  the  total  volume  of  the  system  locus  is  larger  for  doctrine  1  and 
for  doctrine  2: 

E2(l  battery,  doctrine  1)  =  0.55 
E2(l  battery,  doctrine  2)  =  0.50 

When  the  tempo  of  operations  is  fast  (the  threat  moves  rapidly) ,  the  window  of 
opportunity  is  small:  it  is  better  to  make  an  accurate  measurement  of  its 
speed  once  and  then  fire  rapidly  in  sequence  without  taking  time  to  make  new 
estimates,  rather  than  to  make  an  estimate,  shoot,  make  a  new  measurement,  and 
so  on.  Therefore,  the  LSS  doctrine  is  more  effective  than  the  LSLS  doctrine. 
The  first  doctrine  provides  a  more  timely  response  —  while  the  initial 
response  time  of  both  doctrines  is  the  same,  doctrine  1  allows  for  a  higher 
rate  of  firing  than  doctrine  2. 

The  Two  Battery  Case 


When  the  two  batteries  Bx  and  B2  are  considered,  it  appears  from  Fig. 
4 .4  that  their  areas  of  coverage  overlap .  The  threat  moves  first  on  a  part  of 
the  road  that  is  covered  by  battery  B1,  then  on  a  part  that  is  covered  by  both 
B2  and  Bj.and  then  on  the  part  that  is  covered  by  Bt  alone.  It  has  been 
assumed  that  the  tank  has  to  go  beyond  the  range  of  Ba  in  order  to  be  able  to 
fire  at  the  C*  node.  Therefore,  the  probability  of  kill  varies  with  time, 
suddenly  increasing,  then  decreasing.  Assuming  a  "LOOK- SHOOT- SHOOT-SHOOT ... " 
doctrine,  two  different  options  for  the  fire  support  commander  will  be 
considered: 

Option  1:  The  two  batteries  shoot  at  the  threat  independently,  each  one  using 
its  window  of  opportunity  at  the  maximum.  There  is  no  coordination  between 
the  two  batteries . 

Option  2:  Battery  Ba  starts  firing  only  when  the  threat  enters  the  area 
covered  by  both  batteries.  In  other  words,  the  commander  decides  not  to  fire 
Immediately  with  Bx,  but  to  wait  until  coordinated  fire  can  be  achieved,  i.e., 
both  batteries  Bx  and  B,  shooting  so  that  their  projectiles  hit  the  target 
trajectory  at  the  same  time.  The  system's  window  of  opportunity  is  thus 
reduced  to  that  of  battery  Ba.  The  time  Interval  during  which  Bx  holds  its 
fire  can  be  used  to  keep  the  observer's  estimate  updated. 


Figure  4.11  and  4.12  show  the  system  locus  and  its  intersection  (shaded 
region)  with  the  mission  locus  for  both  Options  1  and  2.  The  evaluation  of 
the  effectiveness  of  the  system  for  both  options,  measured  by  the  ratio  of  the 
shaded  volume  over  the  total  volume  of  the  system  locus,  yields  the  following 
results.  Let  E^l)  be  the  MOE  when  Option  1  is  used  and  Ex(2)  when  Option  2 
is  used. 

Then 

EJ1)  £?  Ex(2)  £  0.6 

Therefore,  both  options  result  in  approximately  the  same  value  for  the 
effectiveness  of  the  system.  In  Option  2,  fewer  shots  are  fired  than  in 
Option  1.  In  this  case,  coordination  reduces  costs  for  the  same  kill 
probability.  However,  this  is  not  reflected  in  the  MOE.  Additional  MOPs, 
such  as  cost,  must  be  introduced,  if  one  would  like  to  discriminate  between 
the  two  options. 

It  is  important  to  note  that  Option  2,  coordinated  fire,  is  a  better 
choice  than  Option  1,  although  its  window  of  opportunity  is  much  narrower.  In 
fact,  the  time  available  is  better  managed:  it  is  more  effective  to  wait  in 
order  to  implement  a  better  option.  This  example  shows  that  a  larger  size  of 
the  window  of  opportunity  does  not  necessarily  lead  to  higher  effectiveness. 


Figure  4.11  Option  1  (immediate  fire  without  coordination) 

System  and  Mission  Loci 


o  100  200  300  400  500  600  At 


Figure  4.12.  Option  2  (wait  and  coordinate)  System  and  Mission  Loci 
4.7  CONCLUSIONS 

In  this  chapter,  the  notion  of  timeliness  has  been  integrated  in  the 
analysis  of  a  system's  effectiveness  by  treating  the  temporal  characteristics 
of  the  system  as  performance  measures.  The  resulting  measures  of 
effectiveness  have  embedded  in  them  the  time-related  notions  of  response  time 
and  tempo  of  operations  through  their  impact  on  the  window  of  opportunity. 
These  MOEs  allow  also  the  quantitative  comparison  of  different  doctrines: 
Some  doctrines  are  shown  to  make  better  use  of  the  available  time  than  others . 
Therefore,  this  methodology  for  effectiveness  analysis  can  be  used  to  evaluate 
doctrines  appropriate  to  a  given  situation. 

A  second  point  has  been  illustrated  by  considering  the  relationship 
between  system  components.  While  the  speed  of  processing  and  transmission  of 
data  can  be  improved,  the  effectiveness  of  the  system  may  not  change  if,  for 
instance,  the  reliability  and  survivability  of  the  system's  components  are  not 
improved  also.  Faster  does  not  necessarily  mean  better;  it  can  even  mean 
worse,  if  the  increase  in  speed  is  gained  at  the  expenses  of  the  system's 
survivability.  The  methodology  allows  one  to  relate  a  change  in  one  part  of 
the  system  to  a  change  in  another  part:  the  variations  in  the  features,  of  a 
given  system  are  not  considered  separately,  but  jointly.  This  yields  useful 
perspectives  for  the  design  of  future  C*  systems.  The  influence  of  any 
modification  either  in  the  components,  or  the  structure,  or  the  doctrine,  can 
be  evaluated,  using  the  proposed  measures  of  effectiveness  of  the  system. 
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A  third  point  refers  to  the  window  of  opportunity.  The  size  of  the 
window  of  opportunity  is  not  a  sufficient  determinant  of  a  system’s 
timeliness.  The  location  of  the  window,  determined  by  the  response  time,  must 
also  be  considered. 

The  methodology  is  flexible  enough  to  be  adapted  to  many  kinds  of 
systems:  evolving  systems  [29],  automotive  [30],  or  manufacturing  [31. 
Current  efforts  are  directed  toward  developing  the  properties  of  several 
classes  of  MOEs,  developing  graphics  software  for  the  efficient  construction 
and  analysis  of  the  system  and  mission  loci,  and  applying  the  methodology  to 
more  complex  problems  such  as  evaluating  the  effectiveness  of  a  large  C* 
system  for  doing  indirect  identification  of  friend,  foe,  or  neutral. 


5.  CONCLUSIONS 


The  goals  of  the  research  reported  in  this  document  were:  (a)  the 
further  development  of  a  quantitative  methodology  for  the  design  analysis,  and 
evaluation  of  command  and  control  organizations,  and  (b)  the  development  of  a 
methodology  for  the  assessment  of  C*  systems  that  is  consistent  with  the 
methodology  for  the  evaluation  of  the  decisionmaking  organizations  these 
systems  support. 

5.1  ORGANIZATIONS 

The  specific  objective  designed  to  meet  the  first  goal  was  the 
consideration  of  the  effect  that  different  subjective  probabilities  and 
perceptions  of  the  value  of  the  tasks  by  individual  decisionmakers  can  have  on 
the  organization's  performance.  The  original  approach  included  the 
development  of  the  counterparts  to  N-dimensional  information  theory  and  the 
Partition  Law  of  Information  when  weighted  entropy  is  used  in  the  place  of 
classical  entropy.  While  this  development  was  carried  out,  the  formal  results 
did  not  yield  any  insight  into  the  research  questions.  The  resulting 
expressions  were  too  complicated  to  be  amenable  to  a  physical  interpretation. 
Consequently,  the  research  effort  was  redirected  toward  using  the  conventional 
formulation  of  the  Partition  Law  of  Information,  but  complementing  these 
results  with  efficient  computational  procedures  for  the  study  of  the  phenomena 
of  interest . 

The  problem  of  qualitative  differences  in  the  tasks  of  an  organziation 
has  been  approached  by  distinguishing  four  cases: 

Case  1 :  The  organization  designer  assumes  that  the  decisionmakers  know 
the  objective  probabilities  of  the  tasks  to  be  performed  and  all  tasks 
are  of  equal  importance.  In  this  case,  the  weighting  coefficients  in  the 
entropy  expressions  and  in  the  objective  function  are  unity.  This  is  the 
base  case  that  has  been  analyzed  fully  in  earlier  work. 

Case  2:  The  organization  designer  considers  that  the  decisionmakers  do 
not  know  the  correct  probability  distribution  for  the  various  tasks.  In 
this  case,  weighted  entropy  is  used  to  estimate  workload,  but  the 
objective  function  has  unity  weighting  coefficients,  as  in  Case  1. 

Case  3 :  The  designer  assumes  that  the  decisionmakers  do  know  the  actual 
task  probabilities,  but  have  a  different  perception  of  the  relative 
importance  of  tasks.  In  this  case,  the  computation  of  workload  is  as  in 
Case  1,  but  the  objective  function  is  different. 

Case  4 :  This  is  the  most  general  case.  The  designer  assumes  that  the 
decisionmakers  do  not  know  the  actual  task  probabilities  and  give 
different  relative  weights  to  the  various  tasks.  This  case  subsumes  the 
other  three  cases. 


Detailed  analytical  results  for  the  difference  in  workload  under  the 
correct  probability  distribution  and  the  perceived  ones  were  obtained  for  the 
interacting  decisionmaker  with  bounded  rationality.  The  expressions  for  the 
individual  components  of  workload,  i.e.,  throughput,  blockage,  coordination, 
and  noise  were  also  obtained.  However,  the  complexity  of  the  resulting 
expressions  made  it  difficult  to  obtain  general  results.  Therefore,  a 
previously  developed  algorithm  for  three  member  organizations  was  modified  and 
expanded  to  handle  cases,  2,  3,  and  4.  Computational  results  for  two 
alternative  three  person  organizations  were  obtained  using  a  new 
implementation  of  the  algorithm.  This  implementation  was  on  an  IBM  PC/AT  and 
uses  graphics  extensively. 

In  order  to  evaluate  the  effect  that  differences  in  the  perception  of 
task  uncertainty  (Case  2)  have  on  organizational  performance,  a  consistency 
measure  was  introduced.  This  measures  the  effect  of  selecting  strategies  from 
a  perceived  feasible  strategy  space  as  opposed  to  selecting  one  from  the 
actual  feasible  strategy  space. 

5.2  MEASURES  OF  EFFECTIVENESS 

The  basis  for  the  analysis  and  evaluation  of  alternative  organizational 
designs  was  the  performance-workload  locus,  which  is  a  graphical  depiction  of 
the  values  the  measures  of  performances  of  a  system  can  take,  when  all  the 
possible  operating  points  are  considered.  This  locus  represents,  therefore, 
the  system  capabilities.  The  concept  can  be  generalized  when  instead  of 
performance  and  workload,  any  measures  of  performance  (MOPs)  are  considered. 
The  locus  then  becomes  the  system  locus.  Similarly,  the  design  requirements 
can  be  expressed  in  the  performance  space.  In  the  case  of  organizations, 
performance  as  measured  by  accuracy  and  workload  were  the  pertinent  MOPs . 
Consequently,  the  corresponding  requirements  were  for  satisficing  performance 
and  for  workload  less  than  the  bounded  rationality  constraint.  In  the  case  of 
C*  systems,  the  requirements  define  another  locus,  the  requirements  locus  Lr. 

Measures  of  effectiveness  (MOEs)  are  then  quantities  obtained  by 
determining  how  well  a  system  satisfies  the  requirements.  In  the  case  of 
organizations,  the  consistency  measure  Q  was  used  as  an  MOE. 

To  illustrate  the  methodology,  a  generic  version  of  the  C*  system  for 
surface-to-air  missile  batteries  was  used .  One  novel  aspect  of  the  research 
was  the  modeling  and  analysis  of  timeliness  as  it  pertains  to  C*  systems. 

5 .3  PUBLICATIONS 

The  research  results  have  been  documented  in  full  in  thesis  reports  and 
technical  papers.  The  pertinent  publications  are  listed  below: 

BeJJani,  G.  J.,  (1985).  Information  storage  and  access  in  decision  making 
organizations.  M.S.  Thesis,  LIDS-TH-1434,  Laboratory  for  Information  and 
Decision  Systems,  Massachusetts  Institute  of  Technology,  Cambridge, 
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